-- Logs begin at Mon 2022-07-25 05:02:53 UTC, end at Mon 2022-07-25 06:19:33 UTC. -- Jul 25 05:39:52.100834 dv-cinder-848418-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 25 05:39:52.110620 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 25 05:39:52 2022] *** Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: nodename: dv-cinder-848418-2-iscsi Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: machine: x86_64 Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: clock source: unix Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: pcre jit disabled Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: detected number of CPU cores: 4 Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: current working directory: / Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: detected binary path: /usr/bin/uwsgi-core Jul 25 05:39:52.112662 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: your processes number limit is 47680 Jul 25 05:39:52.113325 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: your memory page size is 4096 bytes Jul 25 05:39:52.113325 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: detected max file descriptor number: 2048 Jul 25 05:39:52.113325 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: lock engine: pthread robust mutexes Jul 25 05:39:52.113325 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: thunder lock: enabled Jul 25 05:39:52.113325 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 25 05:39:52.113325 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 25 05:39:52.157478 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: Python main interpreter initialized at 0x558018ced9c0 Jul 25 05:39:52.157478 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: python threads support enabled Jul 25 05:39:52.157478 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: your server socket listen backlog is limited to 100 connections Jul 25 05:39:52.157478 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: your mercy for graceful operations on workers is 90 seconds Jul 25 05:39:52.157937 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: mapped 403077 bytes (393 KB) for 2 cores Jul 25 05:39:52.157937 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: *** Operational MODE: preforking *** Jul 25 05:39:52.158159 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: *** uWSGI is running in multiple interpreter mode *** Jul 25 05:39:52.158159 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: spawned uWSGI master process (pid: 86783) Jul 25 05:39:52.158364 dv-cinder-848418-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 25 05:39:52.159441 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: spawned uWSGI worker 1 (pid: 86791, cores: 1) Jul 25 05:39:52.159441 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: spawned uWSGI worker 2 (pid: 86792, cores: 1) Jul 25 05:39:52.159441 dv-cinder-848418-2-iscsi devstack@keystone.service[86783]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 25 05:39:57.625990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:39:57.625990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: from cryptography.utils import int_from_bytes Jul 25 05:39:57.626781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:39:57.626781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: from cryptography.utils import int_from_bytes Jul 25 05:39:57.627232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:39:57.627232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: from cryptography.utils import int_from_bytes Jul 25 05:39:57.627365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 25 05:39:57.627365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: from cryptography.utils import int_from_bytes Jul 25 05:39:57.675009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 05:39:57.675212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 05:39:57.675397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] command line args: [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 05:39:57.675490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 25 05:39:57.675669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 25 05:39:57.675747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 05:39:57.675844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ================================================================================ {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 05:39:57.675933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] command line args: [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 25 05:39:57.676101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] admin_token = **** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.676212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 25 05:39:57.676324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ================================================================================ {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 25 05:39:57.676409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] config_dir = [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.676547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.676635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] admin_token = **** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.676714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] config_source = [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.676817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] config_dir = [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.676901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] debug = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] config_source = [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] debug = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.677939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.678034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.678315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.678507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.678654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] list_limit = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.678823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.678995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] log_config_append = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] list_limit = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] log_config_append = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] log_dir = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] log_file = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] log_dir = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.679966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] log_file = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.680824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.681962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.682039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.682152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.682239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.682362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.682477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.682607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.682737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.683039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.683613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.683798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] publish_errors = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.684043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.684236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.684317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.684412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] publish_errors = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.684497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.684662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.684747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.684920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] standard_threads = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.685976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] standard_threads = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] use_journal = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] use_json = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] use_stderr = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] use_syslog = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] use_journal = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.686959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] use_json = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.687146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.687303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] use_stderr = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.687383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] use_syslog = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.687460 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.687576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 25 05:39:57.687655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.687845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.687931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.688136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.688228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.688403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.688481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.688659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.688748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.688917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.688994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.689174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.689253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.689435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.689523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.689689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.689766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.689943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.690020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.690198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.690287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.690453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.690539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.690710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.690788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.690969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.691055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.691226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.691302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.691485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.691560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.691747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.691825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.692010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.692091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.692280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.692368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.692552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.692631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.692822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.692899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.693074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.693151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.693326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.693413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.693556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.693635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.693796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.693875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.694045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.694130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.694290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.694367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.694556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.694634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.694810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.694895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.695058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.695134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.695298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.695375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.695576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.695660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.695832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.695909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.696077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.696251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.696567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.696700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.696811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.696905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697109 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.697983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.698070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.698219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.698307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.698454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.698533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.698689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.699479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.699631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.699722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.699722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.700947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.701115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.701217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.701354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.701443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.701607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.701686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.701842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.701962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.702082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.702161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.702326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.702413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.702564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.702643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.702796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.702886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703508 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.703969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.704047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.704221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.704313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.704455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.704541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.704708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.704786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.704939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705176 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705645 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.705947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.706098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.706178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.706341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.706420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.706576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.706656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.706811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.706889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.707972 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.708058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.708231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.708326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.708466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.708544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.708702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.708793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.708938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.709957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.710104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.710191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.710333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.710412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.710563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.710643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.710791 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.710877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711567 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.711949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.712028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.712202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.712297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.712444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.712528 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.712688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.712767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.712928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] auth.external = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.713972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.714124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] auth.external = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.714231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] auth.password = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.714381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.714530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] auth.token = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.714656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.714788 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.714870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.714982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] auth.password = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] auth.token = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.715997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.716085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.716255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] credential.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.716354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.716502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.716581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.716744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.716833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.716974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.717052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.717208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] credential.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.717286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.717445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.717531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.717678 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.717758 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.717927 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.718999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.719152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.719230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.719396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.719483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.719634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.719709 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.719877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.719956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.720127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] federation.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.720235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.720366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.720505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.720610 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.720751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.720861 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] federation.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721484 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.721976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.722056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.722211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.722290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.722459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.722547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.722698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.722777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.722931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723645 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.723965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.724162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.724247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.724378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.724458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.724627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.724719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.724865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.724945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.725101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.725181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.725391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.725477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.725623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.725702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.725855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.725933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.726099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.726186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.726345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.726426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.726592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.726671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.726834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.726920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.727070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.727149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.727317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.727398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.727558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.727638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.727795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.727882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.728988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.729068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.729226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.729305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.729480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.729603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.729729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.729809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.729920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.730970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.731126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.731205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.731362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.731442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.731590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.731669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.731826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.731906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.732060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.732186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.732322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.732401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.732556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.732638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.732785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.732865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.733960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734280 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.734975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.735124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.735209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.735358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.735439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.735601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.735679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.735831 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.735909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.736063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.736270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.736356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.736429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.736548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.736631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.736792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.736873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737711 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.737937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738176 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.738966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.739115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.739195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.739340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.739418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.739582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.739663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.739815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.739895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.740048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.740147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.740305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.740388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.740538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.740617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.740779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.740867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.741961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.742978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.743130 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.743215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.743359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.743439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.743594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.743691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.743825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] resource.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.743908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.744071 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.744197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.744338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.744430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.744570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.744648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.744808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.744888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.745047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] resource.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.745135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.745277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.745356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.745520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.746084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.746270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.746358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.746522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.746602 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.746817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] role.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.746895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.747064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] role.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.747151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.747295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.747439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.747540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.747620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.747781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.747870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.748017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] role.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.748097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.748288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.748369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] role.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.748519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.748611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.748766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.748846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.749965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.750081 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.750210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.750329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.750451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.750559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.750683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.750848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.750963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.751082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.751189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.751329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.751450 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.751582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.751690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.751829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.752021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.752165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.752284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.752385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.752541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.752648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.752791 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.752896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.753935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.754968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.755114 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.755195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.755348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] token.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.755428 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.755581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.755662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.755827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.755909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.756063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.756190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.756325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.756405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.756565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] token.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.756648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.756799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.756878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757119 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.757987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.758068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.758220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.758299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.758461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.758539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.758704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.758784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.758934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.connection = **** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.759985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.760151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.760249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.760398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.760480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.760634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.760714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.760866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.760956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.connection = **** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.761099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.761189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.761903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.762026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.762143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.762368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.762534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.762613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.762695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.762842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.762924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.763086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.763255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.763410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.763550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.763688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.763824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.763960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.764106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.764271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.764410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.764546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.764682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.764819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.764967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.764967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86792) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 05:39:57.765245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.765245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.765245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.765673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.765673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.765950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 25 05:39:57.766085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86791) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 25 05:39:57.795760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.795985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.796236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.796327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.796567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.796797 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.804677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.805148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.833529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.833672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.833744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.833821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.975331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:57.975991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.024762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.024991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.025079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.025226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.089760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.090912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.091052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.091135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.116123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.116376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.116469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.116602 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.116687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.116827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.116910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.117048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.117130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.117256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.117336 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.117492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.117576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.117714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.129256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.129394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.142999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.143091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.143213 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.143296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.143424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86792) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.143507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86791) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 25 05:39:58.161073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.161285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:39:58.161465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.161621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.161709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:39:58.161823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:39:58.161994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.162079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.162195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 25 05:39:58.162342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.162466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.162601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.162724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.162855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.162977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.163227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.174642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.174928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.175072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.175197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.175346 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.175476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.175613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.175892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.183769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.184081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.187082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.187359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.187445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.187634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.187718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.187912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.188026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.188325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.188411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.188592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.188674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.188852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.188947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.189117 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.189198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.189386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.189467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.189651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.189735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.189918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.189998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.190181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.217188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.217683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.218799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.219080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.219290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.219570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.224378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.224661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.224807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.224944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.225083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.225401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.232942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.233225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.233312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.233564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.237867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.238155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.238260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.238437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.238520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.238713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.238804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.238991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.239080 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.239260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.239343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.239528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.239611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.239791 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.239873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.240055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.258474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.258613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.258750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.258886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.263388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.263632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.267135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.267283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.267432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.267576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.272480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.272584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.276132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.276220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.280463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.280757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.281012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.281098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.281310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.281394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.281584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.281679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.281859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.281945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.282146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.282424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.301524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.301895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.302163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.302247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.302424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.302540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.302825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.303097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.310417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.311241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.312106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.312401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.312990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.313266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.315425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.315704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.315982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.316285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.316397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.316586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.316686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.316951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.317234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.317526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.330427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.330702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.330980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.331267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.331352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.331549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.331634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.331830 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.331915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.332100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.332207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.332409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.332696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.332969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.352170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.352937 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.356011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.356781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.359319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.360080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.360944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.361710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.362567 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.362851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.363136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.363343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.363629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.363919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.370991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.371483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.372646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.373152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.375862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.376167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.376404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.376491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.376699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.376792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.376984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.377270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.385908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.386215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.386491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.386620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.386913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.387193 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.394339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86792) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.394616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.394893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.395094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86791) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 25 05:39:58.395178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.395374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.395458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.395655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.395747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.395932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.396016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.396238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.396325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.396515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.396599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.396782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.396865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.397049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.397140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.397307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.397390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.397579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.397660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.397846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86792) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.397929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.398104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.398382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.398659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86791) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 25 05:39:58.437148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558018ced9c0 pid: 86792 (default app) Jul 25 05:39:58.438152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558018ced9c0 pid: 86791 (default app) Jul 25 05:39:58.440462 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcc643-6fe4-4e79-b996-c6c9304d1cf3 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:39:58.440760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcc643-6fe4-4e79-b996-c6c9304d1cf3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:39:58.441040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcc643-6fe4-4e79-b996-c6c9304d1cf3 None None] PATH_INFO: `/v3` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:39:58.445236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Mon Jul 25 05:39:58 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:06.922534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6a390-b20a-4aab-9050-dd7511944b2f None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.923096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6a390-b20a-4aab-9050-dd7511944b2f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.923096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6a390-b20a-4aab-9050-dd7511944b2f None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:06.927422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:06.933247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:06.940929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:06.940929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.005972 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: INFO dbcounter [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] Registered counter for database keystone Jul 25 05:40:07.027490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG oslo_db.sqlalchemy.engines [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86792) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 05:40:07.034631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writer thread running {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 05:40:07.215313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG passlib.handlers.bcrypt [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86792) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 05:40:07.215745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG passlib.handlers.bcrypt [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86792) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 05:40:07.269486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:07.271114 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:07.392931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 25 05:40:07.600279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6a390-b20a-4aab-9050-dd7511944b2f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:07.670424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: INFO dbcounter [None req-c2a6a390-b20a-4aab-9050-dd7511944b2f None None] Registered counter for database keystone Jul 25 05:40:07.689328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG oslo_db.sqlalchemy.engines [None req-c2a6a390-b20a-4aab-9050-dd7511944b2f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86791) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 25 05:40:07.695508 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writer thread running {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 25 05:40:07.814327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6a390-b20a-4aab-9050-dd7511944b2f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:07.818625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ff81c-6fc8-42b9-8397-6a80449a5262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:07.819289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ff81c-6fc8-42b9-8397-6a80449a5262 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:07.819575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ff81c-6fc8-42b9-8397-6a80449a5262 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:07.819854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ff81c-6fc8-42b9-8397-6a80449a5262 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.828853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ff81c-6fc8-42b9-8397-6a80449a5262 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:07.880635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3d3ff81c-6fc8-42b9-8397-6a80449a5262 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 05:40:07.880976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ff81c-6fc8-42b9-8397-6a80449a5262 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:07.891172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Mon Jul 25 05:40:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 05:40:07.897280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:07.911902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00179e32-abe2-4e82-b6a3-4f3a64d74264 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:07.916321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76caa61-812e-4147-a4f9-bcdc18fb8a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:07.917479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76caa61-812e-4147-a4f9-bcdc18fb8a16 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:07.917957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76caa61-812e-4147-a4f9-bcdc18fb8a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:07.918283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76caa61-812e-4147-a4f9-bcdc18fb8a16 admin admin] PATH_INFO: `/v3/projects/1d6ac2b8c3194197aedc989787229928` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:07.922435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76caa61-812e-4147-a4f9-bcdc18fb8a16 admin admin] RBAC: Authorizing `identity:get_project(project_id=1d6ac2b8c3194197aedc989787229928)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:07.973655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f76caa61-812e-4147-a4f9-bcdc18fb8a16 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 25 05:40:07.974188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76caa61-812e-4147-a4f9-bcdc18fb8a16 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:07.978557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Mon Jul 25 05:40:07 2022] GET /identity/v3/projects/1d6ac2b8c3194197aedc989787229928 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:10.197542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1dfc3-8ce5-4e0d-bcd4-f28a464c515a None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.198066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1dfc3-8ce5-4e0d-bcd4-f28a464c515a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.198163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1dfc3-8ce5-4e0d-bcd4-f28a464c515a None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.202719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:10.208119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c64ab8-9fbc-496d-a1b0-a7e98fc9f949 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.208547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c64ab8-9fbc-496d-a1b0-a7e98fc9f949 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.208692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c64ab8-9fbc-496d-a1b0-a7e98fc9f949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.227880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d1c64ab8-9fbc-496d-a1b0-a7e98fc9f949 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:10.229440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d1c64ab8-9fbc-496d-a1b0-a7e98fc9f949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:10.238049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 25 05:40:10.259947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1dfc3-8ce5-4e0d-bcd4-f28a464c515a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:10.271820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1dfc3-8ce5-4e0d-bcd4-f28a464c515a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:10.275981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b7c771-8d26-4724-a8c7-458a3d8d9526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:10.276634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7c771-8d26-4724-a8c7-458a3d8d9526 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.277452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7c771-8d26-4724-a8c7-458a3d8d9526 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.277728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b7c771-8d26-4724-a8c7-458a3d8d9526 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.285484 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7c771-8d26-4724-a8c7-458a3d8d9526 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:10.287246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b7c771-8d26-4724-a8c7-458a3d8d9526 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:10.296502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Mon Jul 25 05:40:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:40:10.301890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c64ab8-9fbc-496d-a1b0-a7e98fc9f949 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:10.311147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c64ab8-9fbc-496d-a1b0-a7e98fc9f949 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:10.315417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c0cf7-bb85-4fdb-a0d6-81bd17d4e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:10.316301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c0cf7-bb85-4fdb-a0d6-81bd17d4e5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:10.316588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c0cf7-bb85-4fdb-a0d6-81bd17d4e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:10.316866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c0cf7-bb85-4fdb-a0d6-81bd17d4e5f8 admin admin] PATH_INFO: `/v3/users/2df7d0fe893f4cada8ecf1e428b8814a` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:10.318253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c0cf7-bb85-4fdb-a0d6-81bd17d4e5f8 admin admin] RBAC: Authorizing `identity:get_user(user_id=2df7d0fe893f4cada8ecf1e428b8814a)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:10.320062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c0cf7-bb85-4fdb-a0d6-81bd17d4e5f8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:10.321645 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Mon Jul 25 05:40:10 2022] GET /identity/v3/users/2df7d0fe893f4cada8ecf1e428b8814a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:12.400845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd39b0-d596-4388-bc18-af3910c44f74 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.401403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd39b0-d596-4388-bc18-af3910c44f74 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.401403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd39b0-d596-4388-bc18-af3910c44f74 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.402886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:12.407979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dfae6-2bdb-48a0-92f7-431bad1b0113 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.408428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dfae6-2bdb-48a0-92f7-431bad1b0113 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.408584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3dfae6-2bdb-48a0-92f7-431bad1b0113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.425074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4f3dfae6-2bdb-48a0-92f7-431bad1b0113 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:12.426263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4f3dfae6-2bdb-48a0-92f7-431bad1b0113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:12.445680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943afa5-5a4b-4c97-88fd-bd3e4649e404 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.445977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943afa5-5a4b-4c97-88fd-bd3e4649e404 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.446263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943afa5-5a4b-4c97-88fd-bd3e4649e404 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.447738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:12.453578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.453869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.454151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.492784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG passlib.handlers.bcrypt [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86791) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 25 05:40:12.493196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG passlib.handlers.bcrypt [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86791) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 25 05:40:12.510747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:12.533881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dfae6-2bdb-48a0-92f7-431bad1b0113 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.539471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:12.541110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:12.556832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.638054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3dfae6-2bdb-48a0-92f7-431bad1b0113 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.641545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:12.663031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.671819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811a44-23f0-48e6-9dff-6882a4b3b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.672658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811a44-23f0-48e6-9dff-6882a4b3b688 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.672948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811a44-23f0-48e6-9dff-6882a4b3b688 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.673235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d811a44-23f0-48e6-9dff-6882a4b3b688 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.685321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d811a44-23f0-48e6-9dff-6882a4b3b688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-4d811a44-23f0-48e6-9dff-6882a4b3b688 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:12.696681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:12.697771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:12.699701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:12.706611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811a44-23f0-48e6-9dff-6882a4b3b688 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.763922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a727203-a7f2-4f07-85de-9b4a47e6f015 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.766289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.791907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:12.802328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bccd1b-62ab-4902-a163-deeac8931794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.803256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bccd1b-62ab-4902-a163-deeac8931794 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.803547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bccd1b-62ab-4902-a163-deeac8931794 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.803829 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bccd1b-62ab-4902-a163-deeac8931794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.811242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d811a44-23f0-48e6-9dff-6882a4b3b688 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.845664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:12.846330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:12.846616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:12.846895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:12.847653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:12.847964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:12.849413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:12.861882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:12.867386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.877483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bccd1b-62ab-4902-a163-deeac8931794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:12.879015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bccd1b-62ab-4902-a163-deeac8931794 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:12.966066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f04c69-8448-4212-b5be-2374170b20be None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:12.975423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:12.981828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bccd1b-62ab-4902-a163-deeac8931794 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:12.984492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.000214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea8e12-28c8-4fab-860e-0c45dd2f1d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.001120 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea8e12-28c8-4fab-860e-0c45dd2f1d7d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.001461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea8e12-28c8-4fab-860e-0c45dd2f1d7d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.001784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea8e12-28c8-4fab-860e-0c45dd2f1d7d None admin] PATH_INFO: `/v3/users/2df7d0fe893f4cada8ecf1e428b8814a` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.003025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea8e12-28c8-4fab-860e-0c45dd2f1d7d None admin] RBAC: Authorizing `identity:get_user(user_id=2df7d0fe893f4cada8ecf1e428b8814a)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.004557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea8e12-28c8-4fab-860e-0c45dd2f1d7d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.006198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/users/2df7d0fe893f4cada8ecf1e428b8814a => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:13.011433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea8e12-28c8-4fab-860e-0c45dd2f1d7d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.025434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.082168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bccd1b-62ab-4902-a163-deeac8931794 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.111248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea8e12-28c8-4fab-860e-0c45dd2f1d7d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.115502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.116329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.116620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.116902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.135687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.137222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.145129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3324-b2e1-493f-b94e-183e56f20702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.145924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3324-b2e1-493f-b94e-183e56f20702 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.146211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3324-b2e1-493f-b94e-183e56f20702 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.146490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba3324-b2e1-493f-b94e-183e56f20702 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.147670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba3324-b2e1-493f-b94e-183e56f20702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.149137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba3324-b2e1-493f-b94e-183e56f20702 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.150735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:13.157083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3324-b2e1-493f-b94e-183e56f20702 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:13.158685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:13.159552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:13.160430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:13.160430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:13.160430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:13.160430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:13.160430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:13.166234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:13.224061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.238061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:13.257039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba3324-b2e1-493f-b94e-183e56f20702 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.268608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62163411-56af-4768-9003-60ab7ce091e6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:13.291468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079673d-194f-4eaf-a406-15ee370afcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.292245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079673d-194f-4eaf-a406-15ee370afcd6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.293043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079673d-194f-4eaf-a406-15ee370afcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.293395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079673d-194f-4eaf-a406-15ee370afcd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.294168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079673d-194f-4eaf-a406-15ee370afcd6 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.domain.id=default, user.id=2df7d0fe893f4cada8ecf1e428b8814a) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:13.294477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079673d-194f-4eaf-a406-15ee370afcd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.295933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079673d-194f-4eaf-a406-15ee370afcd6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.302126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9b5c0-28f3-40bb-a1e6-a4aff46b85ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:13.302537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1322 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/role_assignments?user.id=2df7d0fe893f4cada8ecf1e428b8814a&scope.domain.id=default&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:13.302908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9b5c0-28f3-40bb-a1e6-a4aff46b85ec None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:13.303203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9b5c0-28f3-40bb-a1e6-a4aff46b85ec None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:13.303484 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9b5c0-28f3-40bb-a1e6-a4aff46b85ec None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:13.304330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9b5c0-28f3-40bb-a1e6-a4aff46b85ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:13.304643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9b5c0-28f3-40bb-a1e6-a4aff46b85ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:13.306061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9b5c0-28f3-40bb-a1e6-a4aff46b85ec None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:13.327032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:15.776511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197c123-b230-4ff7-a74f-9dfd4727856d None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.777544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197c123-b230-4ff7-a74f-9dfd4727856d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.778033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197c123-b230-4ff7-a74f-9dfd4727856d None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.779643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:15.785316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b2cc-ab08-4a4a-81e7-6b6412e30ecd None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.785737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b2cc-ab08-4a4a-81e7-6b6412e30ecd None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.785948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79b2cc-ab08-4a4a-81e7-6b6412e30ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.817148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9f79b2cc-ab08-4a4a-81e7-6b6412e30ecd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:15.818471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9f79b2cc-ab08-4a4a-81e7-6b6412e30ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:15.848634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5f8ac-51c7-478d-8169-d76450e94005 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.849418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5f8ac-51c7-478d-8169-d76450e94005 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.861983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5f8ac-51c7-478d-8169-d76450e94005 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.861983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 13/23] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:15.861983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af136927-7fd3-44ee-a44c-fff0d71648b5 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:15.861983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af136927-7fd3-44ee-a44c-fff0d71648b5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:15.861983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af136927-7fd3-44ee-a44c-fff0d71648b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:15.895202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-af136927-7fd3-44ee-a44c-fff0d71648b5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:15.896495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-af136927-7fd3-44ee-a44c-fff0d71648b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:15.918608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 11/24] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:15.958384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b2cc-ab08-4a4a-81e7-6b6412e30ecd None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.021227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 14/25] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:16.028991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:16.046008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af136927-7fd3-44ee-a44c-fff0d71648b5 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.128752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:16.147975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79b2cc-ab08-4a4a-81e7-6b6412e30ecd None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:16.156712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af136927-7fd3-44ee-a44c-fff0d71648b5 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:16.195310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb84d92-d2f3-48f7-ac74-a1b527267cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:16.196426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb84d92-d2f3-48f7-ac74-a1b527267cfa None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.196859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb84d92-d2f3-48f7-ac74-a1b527267cfa None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.197323 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb84d92-d2f3-48f7-ac74-a1b527267cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.210168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb84d92-d2f3-48f7-ac74-a1b527267cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:16.212238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb84d92-d2f3-48f7-ac74-a1b527267cfa None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:16.229019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34484e99-4f2d-4245-b7a2-787e0db5297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:16.230240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484e99-4f2d-4245-b7a2-787e0db5297a None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.230534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484e99-4f2d-4245-b7a2-787e0db5297a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.230813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34484e99-4f2d-4245-b7a2-787e0db5297a None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.231588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34484e99-4f2d-4245-b7a2-787e0db5297a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:16.233023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34484e99-4f2d-4245-b7a2-787e0db5297a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-5fb84d92-d2f3-48f7-ac74-a1b527267cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:40:16.236327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:16.237218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:16.238102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:40:16.238102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:40:16.238102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:40:16.238102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:16.239700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 15/26] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:16.241174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cab2b-45ea-4ac2-baec-4fe28ce3ffea None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.241627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cab2b-45ea-4ac2-baec-4fe28ce3ffea None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.242163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45cab2b-45ea-4ac2-baec-4fe28ce3ffea None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.243962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 16/27] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:16.245660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d32c2-d763-43df-8e11-e695f55bb62f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.245948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d32c2-d763-43df-8e11-e695f55bb62f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.246230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d32c2-d763-43df-8e11-e695f55bb62f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.249013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 17/28] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:16.249013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d32c2-d763-43df-8e11-e695f55bb62f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.366104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d32c2-d763-43df-8e11-e695f55bb62f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:16.393334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:16.402904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2c43-10dc-41af-9f20-9974c2b9e45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:16.409051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2c43-10dc-41af-9f20-9974c2b9e45b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.409051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2c43-10dc-41af-9f20-9974c2b9e45b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.409051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2c43-10dc-41af-9f20-9974c2b9e45b None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.409051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d2c43-10dc-41af-9f20-9974c2b9e45b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:16.409051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d2c43-10dc-41af-9f20-9974c2b9e45b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:16.409051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d2c43-10dc-41af-9f20-9974c2b9e45b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:16.430420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 18/29] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:16.430420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9a4a8-675d-4009-8c44-0de9e9e7fa8c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.430420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9a4a8-675d-4009-8c44-0de9e9e7fa8c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.430420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9a4a8-675d-4009-8c44-0de9e9e7fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.445443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-83d9a4a8-675d-4009-8c44-0de9e9e7fa8c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:16.446691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-83d9a4a8-675d-4009-8c44-0de9e9e7fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:16.459822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 12/30] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 25 05:40:15 2022] POST /identity/v3/roles => generated 219 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:16.461652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e702a-c22e-4849-9665-a1827c002b07 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.462130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e702a-c22e-4849-9665-a1827c002b07 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.462647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e702a-c22e-4849-9665-a1827c002b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.496277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c13e702a-c22e-4849-9665-a1827c002b07 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:16.500586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c13e702a-c22e-4849-9665-a1827c002b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:16.581067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 19/31] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:16.583442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9a4a8-675d-4009-8c44-0de9e9e7fa8c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.644083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 13/32] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:16.646887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e702a-c22e-4849-9665-a1827c002b07 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.710885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:16.776153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9a4a8-675d-4009-8c44-0de9e9e7fa8c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:16.796388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:16.808022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e702a-c22e-4849-9665-a1827c002b07 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:16.815687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a4954-218a-471e-93f6-57aed0b430df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:16.816414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4954-218a-471e-93f6-57aed0b430df None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.816707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4954-218a-471e-93f6-57aed0b430df None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.816990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4954-218a-471e-93f6-57aed0b430df None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.817765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a4954-218a-471e-93f6-57aed0b430df None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:16.819482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a4954-218a-471e-93f6-57aed0b430df None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:16.855254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f266f4-9eb1-460e-9b51-30b46ca0fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:16.856164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f266f4-9eb1-460e-9b51-30b46ca0fd7a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:16.856463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f266f4-9eb1-460e-9b51-30b46ca0fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:16.856748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f266f4-9eb1-460e-9b51-30b46ca0fd7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:16.871676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f266f4-9eb1-460e-9b51-30b46ca0fd7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-f9f266f4-9eb1-460e-9b51-30b46ca0fd7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:16.880826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:16.881698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:16.882448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 14/33] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:16.883110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f266f4-9eb1-460e-9b51-30b46ca0fd7a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:16.995620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.002025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f266f4-9eb1-460e-9b51-30b46ca0fd7a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.039857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2df9c-dbb1-489f-bf2f-b9f1df235b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.040594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2df9c-dbb1-489f-bf2f-b9f1df235b39 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.040879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2df9c-dbb1-489f-bf2f-b9f1df235b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.041163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2df9c-dbb1-489f-bf2f-b9f1df235b39 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.041920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2df9c-dbb1-489f-bf2f-b9f1df235b39 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.044220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2df9c-dbb1-489f-bf2f-b9f1df235b39 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.073840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 20/34] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:40:16 2022] POST /identity/v3/projects => generated 301 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:17.075404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a4954-218a-471e-93f6-57aed0b430df None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.117400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 15/35] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 25 05:40:16 2022] POST /identity/v3/roles => generated 225 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:17.240802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cef0c9-66ec-491e-b04f-fb6b59f1bac8 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.241099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cef0c9-66ec-491e-b04f-fb6b59f1bac8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.241549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cef0c9-66ec-491e-b04f-fb6b59f1bac8 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.243078 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 16/36] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:17.250167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a4954-218a-471e-93f6-57aed0b430df None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.256837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf523-02f9-4f40-b827-54c98ac712fb None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.257187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf523-02f9-4f40-b827-54c98ac712fb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.257419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf523-02f9-4f40-b827-54c98ac712fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.296606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645253b9-ae7f-40a7-baa7-922c66817037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.297139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-29bbf523-02f9-4f40-b827-54c98ac712fb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:17.297921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645253b9-ae7f-40a7-baa7-922c66817037 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.298371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-29bbf523-02f9-4f40-b827-54c98ac712fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:17.298614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645253b9-ae7f-40a7-baa7-922c66817037 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.299100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645253b9-ae7f-40a7-baa7-922c66817037 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.300070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645253b9-ae7f-40a7-baa7-922c66817037 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:17.300880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645253b9-ae7f-40a7-baa7-922c66817037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.302654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645253b9-ae7f-40a7-baa7-922c66817037 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.316200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 21/37] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:17.318289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522083ca-5975-450d-9a9e-dbe64e4d5f6a None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.318781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522083ca-5975-450d-9a9e-dbe64e4d5f6a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.319070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522083ca-5975-450d-9a9e-dbe64e4d5f6a None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.320821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 22/38] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:17.322900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522083ca-5975-450d-9a9e-dbe64e4d5f6a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.363273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.386388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 17/39] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:17.387801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f4e87-ec82-4480-88e7-d962ff40e93d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.388292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f4e87-ec82-4480-88e7-d962ff40e93d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.388397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f4e87-ec82-4480-88e7-d962ff40e93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.406474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8d7f4e87-ec82-4480-88e7-d962ff40e93d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:17.407694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8d7f4e87-ec82-4480-88e7-d962ff40e93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:17.482719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 18/40] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:17.484814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f4e87-ec82-4480-88e7-d962ff40e93d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.504831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.548392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522083ca-5975-450d-9a9e-dbe64e4d5f6a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.583875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed4625-e6c3-4398-8f64-a2766b9c66b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.584787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed4625-e6c3-4398-8f64-a2766b9c66b2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.585081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed4625-e6c3-4398-8f64-a2766b9c66b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.585363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed4625-e6c3-4398-8f64-a2766b9c66b2 None admin] PATH_INFO: `/v3/users/2df7d0fe893f4cada8ecf1e428b8814a` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.586679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed4625-e6c3-4398-8f64-a2766b9c66b2 None admin] RBAC: Authorizing `identity:get_user(user_id=2df7d0fe893f4cada8ecf1e428b8814a)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.586928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f4e87-ec82-4480-88e7-d962ff40e93d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.588168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed4625-e6c3-4398-8f64-a2766b9c66b2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.589847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 23/41] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/users/2df7d0fe893f4cada8ecf1e428b8814a => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:17.590622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed4625-e6c3-4398-8f64-a2766b9c66b2 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.624362 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bc208-6490-44a9-ad96-7bf1144bf106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.625278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc208-6490-44a9-ad96-7bf1144bf106 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.625569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc208-6490-44a9-ad96-7bf1144bf106 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.625851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6bc208-6490-44a9-ad96-7bf1144bf106 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.626597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bc208-6490-44a9-ad96-7bf1144bf106 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.627990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6bc208-6490-44a9-ad96-7bf1144bf106 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.670099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 19/42] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 25 05:40:17 2022] POST /identity/v3/roles => generated 223 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:17.671006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bc208-6490-44a9-ad96-7bf1144bf106 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.695554 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.700378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed4625-e6c3-4398-8f64-a2766b9c66b2 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.740431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149bd82-3810-4ae6-b5ce-7af3567aa2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.741256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149bd82-3810-4ae6-b5ce-7af3567aa2eb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.741548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149bd82-3810-4ae6-b5ce-7af3567aa2eb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.741828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c149bd82-3810-4ae6-b5ce-7af3567aa2eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.743191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149bd82-3810-4ae6-b5ce-7af3567aa2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.744723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c149bd82-3810-4ae6-b5ce-7af3567aa2eb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.746873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 24/43] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:17.752728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149bd82-3810-4ae6-b5ce-7af3567aa2eb None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.770999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:17.796951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6bc208-6490-44a9-ad96-7bf1144bf106 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.841318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02798b-3af1-4c39-9245-b44ecfcfd929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.841318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02798b-3af1-4c39-9245-b44ecfcfd929 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.841318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02798b-3af1-4c39-9245-b44ecfcfd929 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.842093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02798b-3af1-4c39-9245-b44ecfcfd929 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.843502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02798b-3af1-4c39-9245-b44ecfcfd929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.845441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02798b-3af1-4c39-9245-b44ecfcfd929 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.846965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 20/44] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:17.852428 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02798b-3af1-4c39-9245-b44ecfcfd929 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:17.874440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c149bd82-3810-4ae6-b5ce-7af3567aa2eb None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.909874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b893588-4439-4bbe-8b0b-076b4f730fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.910551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b893588-4439-4bbe-8b0b-076b4f730fb8 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.910842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b893588-4439-4bbe-8b0b-076b4f730fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.911126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b893588-4439-4bbe-8b0b-076b4f730fb8 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.911846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b893588-4439-4bbe-8b0b-076b4f730fb8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.913358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b893588-4439-4bbe-8b0b-076b4f730fb8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:17.958488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02798b-3af1-4c39-9245-b44ecfcfd929 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:17.991181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3f29da-eb46-4bb0-b8f3-036e65f88295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:17.992048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f29da-eb46-4bb0-b8f3-036e65f88295 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:17.992360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f29da-eb46-4bb0-b8f3-036e65f88295 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:17.992656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3f29da-eb46-4bb0-b8f3-036e65f88295 None admin] PATH_INFO: `/v3/domains/default/users/2df7d0fe893f4cada8ecf1e428b8814a/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:17.994849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f29da-eb46-4bb0-b8f3-036e65f88295 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2df7d0fe893f4cada8ecf1e428b8814a, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:17.996969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3f29da-eb46-4bb0-b8f3-036e65f88295 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:18.019206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 25/45] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:40:17 2022] POST /identity/v3/projects => generated 312 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:18.068916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 21/46] 10.222.0.40 () {58 vars in 1361 bytes} [Mon Jul 25 05:40:17 2022] PUT /identity/v3/domains/default/users/2df7d0fe893f4cada8ecf1e428b8814a/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:40:20.057122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9418c-5bcd-4684-ae4f-078b4e26ea21 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.058111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9418c-5bcd-4684-ae4f-078b4e26ea21 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.058194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9418c-5bcd-4684-ae4f-078b4e26ea21 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.059524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 26/47] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:20.064677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0b66-c042-496c-b509-f60fc7e5ecdc None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.065116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0b66-c042-496c-b509-f60fc7e5ecdc None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.065259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db0b66-c042-496c-b509-f60fc7e5ecdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.071252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.141695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5e5b0-917f-473a-a4e7-8317ef7e9908 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.141986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5e5b0-917f-473a-a4e7-8317ef7e9908 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.142272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f5e5b0-917f-473a-a4e7-8317ef7e9908 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.143672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 27/48] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:20.149214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d57e3-2596-481d-ad02-fd0a4bc41cf0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.149507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d57e3-2596-481d-ad02-fd0a4bc41cf0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.149787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d57e3-2596-481d-ad02-fd0a4bc41cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.167570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-996d57e3-2596-481d-ad02-fd0a4bc41cf0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:20.169153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-996d57e3-2596-481d-ad02-fd0a4bc41cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:20.183769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.187744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-35db0b66-c042-496c-b509-f60fc7e5ecdc None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:20.189019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-35db0b66-c042-496c-b509-f60fc7e5ecdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:20.238542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 28/49] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:20.257973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 22/50] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:20.262833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d57e3-2596-481d-ad02-fd0a4bc41cf0 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.281659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db0b66-c042-496c-b509-f60fc7e5ecdc None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.360737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d57e3-2596-481d-ad02-fd0a4bc41cf0 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.380737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db0b66-c042-496c-b509-f60fc7e5ecdc None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.394133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c9ddf-16e2-4ffc-a8c6-09280268a868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.394931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9ddf-16e2-4ffc-a8c6-09280268a868 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.395221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9ddf-16e2-4ffc-a8c6-09280268a868 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.395506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415c9ddf-16e2-4ffc-a8c6-09280268a868 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.405600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415c9ddf-16e2-4ffc-a8c6-09280268a868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-415c9ddf-16e2-4ffc-a8c6-09280268a868 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:20.413254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:20.414081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:20.414850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:20.415412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9f9e-f316-4289-a3bd-aacf66cae59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.415501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 29/51] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:20.415567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9f9e-f316-4289-a3bd-aacf66cae59a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.415567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9f9e-f316-4289-a3bd-aacf66cae59a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.415567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f9f9e-f316-4289-a3bd-aacf66cae59a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.417569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f9f9e-f316-4289-a3bd-aacf66cae59a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.418982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f9f9e-f316-4289-a3bd-aacf66cae59a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.420642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 23/52] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:20.421018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c9ddf-16e2-4ffc-a8c6-09280268a868 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.426044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9f9e-f316-4289-a3bd-aacf66cae59a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.428688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.433280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.524401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415c9ddf-16e2-4ffc-a8c6-09280268a868 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.526825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f9f9e-f316-4289-a3bd-aacf66cae59a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.557356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.558572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.558860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.559181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.559936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:20.560280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.560753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75f5fb-a2df-4eac-ba1d-b91c4a0f2596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.561418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f5fb-a2df-4eac-ba1d-b91c4a0f2596 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.561687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.561771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f5fb-a2df-4eac-ba1d-b91c4a0f2596 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.561971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f5fb-a2df-4eac-ba1d-b91c4a0f2596 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.562678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75f5fb-a2df-4eac-ba1d-b91c4a0f2596 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.564147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75f5fb-a2df-4eac-ba1d-b91c4a0f2596 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.569748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 30/53] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:20.575590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.642965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.673820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ff2b3-1fab-49ac-85c7-1828139dec24 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.675662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 24/54] 10.222.0.40 () {60 vars in 1140 bytes} [Mon Jul 25 05:40:20 2022] POST /identity/v3/projects => generated 298 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:20.712398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77739d66-bb90-4b87-855c-ea39ef867518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.713345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77739d66-bb90-4b87-855c-ea39ef867518 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.714182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77739d66-bb90-4b87-855c-ea39ef867518 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.714455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77739d66-bb90-4b87-855c-ea39ef867518 None admin] PATH_INFO: `/v3/users/2df7d0fe893f4cada8ecf1e428b8814a` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.715785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77739d66-bb90-4b87-855c-ea39ef867518 None admin] RBAC: Authorizing `identity:get_user(user_id=2df7d0fe893f4cada8ecf1e428b8814a)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.717322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77739d66-bb90-4b87-855c-ea39ef867518 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.719057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 31/55] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/users/2df7d0fe893f4cada8ecf1e428b8814a => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:20.725052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75f5fb-a2df-4eac-ba1d-b91c4a0f2596 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.791250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:20.840446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75f5fb-a2df-4eac-ba1d-b91c4a0f2596 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:20.875029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68a3e3-a9d9-443a-a506-6aae2d16708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:20.876021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68a3e3-a9d9-443a-a506-6aae2d16708d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:20.876343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68a3e3-a9d9-443a-a506-6aae2d16708d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:20.877447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68a3e3-a9d9-443a-a506-6aae2d16708d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:20.877951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68a3e3-a9d9-443a-a506-6aae2d16708d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:20.879472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68a3e3-a9d9-443a-a506-6aae2d16708d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:20.881169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 25/56] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:20.891841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77739d66-bb90-4b87-855c-ea39ef867518 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:20.988645 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77739d66-bb90-4b87-855c-ea39ef867518 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:21.022395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c9b39-2182-42f5-958a-3fcf2596f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:21.023054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c9b39-2182-42f5-958a-3fcf2596f476 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:21.023367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c9b39-2182-42f5-958a-3fcf2596f476 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:21.023648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c9b39-2182-42f5-958a-3fcf2596f476 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:21.024518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c9b39-2182-42f5-958a-3fcf2596f476 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.domain.id=default, user.id=2df7d0fe893f4cada8ecf1e428b8814a) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:21.024841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c9b39-2182-42f5-958a-3fcf2596f476 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:21.026399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c9b39-2182-42f5-958a-3fcf2596f476 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:21.032600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 32/57] 10.222.0.40 () {56 vars in 1322 bytes} [Mon Jul 25 05:40:20 2022] GET /identity/v3/role_assignments?user.id=2df7d0fe893f4cada8ecf1e428b8814a&scope.domain.id=default&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:22.732671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a002905-a97f-4c75-a5a4-436bb457b4a6 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.733218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a002905-a97f-4c75-a5a4-436bb457b4a6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.733218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a002905-a97f-4c75-a5a4-436bb457b4a6 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.734603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 26/58] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:22.739707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deae5a9-4092-4735-87bf-12261ec66797 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.740137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deae5a9-4092-4735-87bf-12261ec66797 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.740494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deae5a9-4092-4735-87bf-12261ec66797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.746972 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:22.758938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0deae5a9-4092-4735-87bf-12261ec66797 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:22.760159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0deae5a9-4092-4735-87bf-12261ec66797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:22.826320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 33/59] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:22.850434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a002905-a97f-4c75-a5a4-436bb457b4a6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:22.949947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a002905-a97f-4c75-a5a4-436bb457b4a6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:22.971820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:22.983284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca139c5a-a20c-4509-af09-c370d99dc2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:22.984008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca139c5a-a20c-4509-af09-c370d99dc2ce None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:22.984333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca139c5a-a20c-4509-af09-c370d99dc2ce None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:22.984620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca139c5a-a20c-4509-af09-c370d99dc2ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:22.985799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca139c5a-a20c-4509-af09-c370d99dc2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:22.987290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca139c5a-a20c-4509-af09-c370d99dc2ce None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:22.989039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 27/60] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:22.994457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deae5a9-4092-4735-87bf-12261ec66797 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:23.090453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deae5a9-4092-4735-87bf-12261ec66797 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:23.118931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:23.123803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f4e06-0aeb-42e6-97b9-177f8d94cc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:23.124507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f4e06-0aeb-42e6-97b9-177f8d94cc15 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:23.124798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f4e06-0aeb-42e6-97b9-177f8d94cc15 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:23.125078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f4e06-0aeb-42e6-97b9-177f8d94cc15 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:23.125798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f4e06-0aeb-42e6-97b9-177f8d94cc15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:23.127262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384f4e06-0aeb-42e6-97b9-177f8d94cc15 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:23.210470 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 34/61] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:22 2022] POST /identity/v3/users => generated 275 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:26.214542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fbd4a-3117-4c57-8703-39a9abeb9283 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.215419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fbd4a-3117-4c57-8703-39a9abeb9283 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.215868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fbd4a-3117-4c57-8703-39a9abeb9283 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.217436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 28/62] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:26.222992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba76fb-0841-4aac-9188-3847146ae0fb None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.223776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba76fb-0841-4aac-9188-3847146ae0fb None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.224233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba76fb-0841-4aac-9188-3847146ae0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.249178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-3dba76fb-0841-4aac-9188-3847146ae0fb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:26.250656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3dba76fb-0841-4aac-9188-3847146ae0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:26.342444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 35/63] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:26.385814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fbd4a-3117-4c57-8703-39a9abeb9283 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.539883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fbd4a-3117-4c57-8703-39a9abeb9283 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.587700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107a6a5-364e-458e-92fa-e0c4068aedb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.588921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107a6a5-364e-458e-92fa-e0c4068aedb3 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.589427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107a6a5-364e-458e-92fa-e0c4068aedb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.589901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107a6a5-364e-458e-92fa-e0c4068aedb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.605598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107a6a5-364e-458e-92fa-e0c4068aedb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-2107a6a5-364e-458e-92fa-e0c4068aedb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:26.613674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:26.614509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:26.615257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:26.617599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 29/64] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:26.624048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba76fb-0841-4aac-9188-3847146ae0fb None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.653794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e293-224a-48a2-94c7-502d00c34a69 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.654110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e293-224a-48a2-94c7-502d00c34a69 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.654367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6e293-224a-48a2-94c7-502d00c34a69 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.655886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 30/65] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:26.661068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7deb6-b415-438a-9356-54826df38592 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.661591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7deb6-b415-438a-9356-54826df38592 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.662091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7deb6-b415-438a-9356-54826df38592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.686461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-19c7deb6-b415-438a-9356-54826df38592 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:26.688724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-19c7deb6-b415-438a-9356-54826df38592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:26.749052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:26.781300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 31/66] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:26.783668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12047480-cfbe-4243-a243-d9ed85e4f6ae None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.784699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12047480-cfbe-4243-a243-d9ed85e4f6ae None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.785209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12047480-cfbe-4243-a243-d9ed85e4f6ae None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.787322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 32/67] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:26.795926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65078d2-b849-4bba-b805-62c7cee562a2 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.796185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65078d2-b849-4bba-b805-62c7cee562a2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.796459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65078d2-b849-4bba-b805-62c7cee562a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.814643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba76fb-0841-4aac-9188-3847146ae0fb None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:26.817671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a65078d2-b849-4bba-b805-62c7cee562a2 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:26.818842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a65078d2-b849-4bba-b805-62c7cee562a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:26.834675 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:26.855143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:26.856082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:26.856590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:26.857016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:26.857927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:26.858381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:26.859949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:26.869286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 36/68] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:26.871385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.899403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 33/69] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:26.903229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65078d2-b849-4bba-b805-62c7cee562a2 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:26.975288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b313cc9-8aeb-4e71-81b5-bf5214a9feb0 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.002121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65078d2-b849-4bba-b805-62c7cee562a2 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.008783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1345e-51ea-4b09-bece-40ba6dd4b13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.009590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1345e-51ea-4b09-bece-40ba6dd4b13e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.009877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1345e-51ea-4b09-bece-40ba6dd4b13e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.010157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1345e-51ea-4b09-bece-40ba6dd4b13e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.013128 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.020809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1345e-51ea-4b09-bece-40ba6dd4b13e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-a7a1345e-51ea-4b09-bece-40ba6dd4b13e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:27.029847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:27.030698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:27.031443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 37/70] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:27.032355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf8571-a912-4428-8534-cef45f891d2c None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.032830 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf8571-a912-4428-8534-cef45f891d2c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.032917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf8571-a912-4428-8534-cef45f891d2c None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.034138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 38/71] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:27.035210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1707321-5953-4243-a977-7f630a3a784a None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.035591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1707321-5953-4243-a977-7f630a3a784a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.035961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1707321-5953-4243-a977-7f630a3a784a None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.037210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 39/72] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:27.038051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1707321-5953-4243-a977-7f630a3a784a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.046921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4606e94-ef68-4463-821d-ef0b8cdbee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.047747 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4606e94-ef68-4463-821d-ef0b8cdbee0e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.048033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4606e94-ef68-4463-821d-ef0b8cdbee0e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.048364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4606e94-ef68-4463-821d-ef0b8cdbee0e None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.064692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4606e94-ef68-4463-821d-ef0b8cdbee0e None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.066172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4606e94-ef68-4463-821d-ef0b8cdbee0e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.067855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 34/73] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:26 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:27.068654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4606e94-ef68-4463-821d-ef0b8cdbee0e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.086008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.146246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1707321-5953-4243-a977-7f630a3a784a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.167919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4606e94-ef68-4463-821d-ef0b8cdbee0e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.179609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f03eac-29c9-40ae-9542-663e8d625ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.180442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f03eac-29c9-40ae-9542-663e8d625ffe None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.180733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f03eac-29c9-40ae-9542-663e8d625ffe None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.181012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f03eac-29c9-40ae-9542-663e8d625ffe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.191294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f03eac-29c9-40ae-9542-663e8d625ffe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-c5f03eac-29c9-40ae-9542-663e8d625ffe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:27.199912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:27.201071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:27.201857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 40/74] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:27.202471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a121fa-bd54-45a8-8664-bbf8f25c3213 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.202731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a121fa-bd54-45a8-8664-bbf8f25c3213 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.203019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a121fa-bd54-45a8-8664-bbf8f25c3213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.204606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f5055-d588-4749-80fc-c18c3edce2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.205287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f5055-d588-4749-80fc-c18c3edce2c9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.205589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f5055-d588-4749-80fc-c18c3edce2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.205954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f5055-d588-4749-80fc-c18c3edce2c9 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.206708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f5055-d588-4749-80fc-c18c3edce2c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:27.207040 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f5055-d588-4749-80fc-c18c3edce2c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.208441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f5055-d588-4749-80fc-c18c3edce2c9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.220090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-00a121fa-bd54-45a8-8664-bbf8f25c3213 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:27.220979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 35/75] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:27.221467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-00a121fa-bd54-45a8-8664-bbf8f25c3213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:27.222028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9d33-1c2b-4be7-a544-82b3bd1282b4 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.222317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9d33-1c2b-4be7-a544-82b3bd1282b4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.222595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e9d33-1c2b-4be7-a544-82b3bd1282b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.238686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-0f6e9d33-1c2b-4be7-a544-82b3bd1282b4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:27.239869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0f6e9d33-1c2b-4be7-a544-82b3bd1282b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:27.263857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.289001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 41/76] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:27.293029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a121fa-bd54-45a8-8664-bbf8f25c3213 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.313862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 36/77] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:27.314637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e9d33-1c2b-4be7-a544-82b3bd1282b4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.340178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.395082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a121fa-bd54-45a8-8664-bbf8f25c3213 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.422437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e9d33-1c2b-4be7-a544-82b3bd1282b4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.429800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bcdc7-7cda-4851-b950-a040d8897d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.430464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bcdc7-7cda-4851-b950-a040d8897d1c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.430754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bcdc7-7cda-4851-b950-a040d8897d1c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.431038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61bcdc7-7cda-4851-b950-a040d8897d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.456298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.456969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.457257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.457565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.458324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:27.458643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.460035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.468605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 37/78] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:27.473325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.497072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.499851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bcdc7-7cda-4851-b950-a040d8897d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.501377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61bcdc7-7cda-4851-b950-a040d8897d1c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.532842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.570790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf865c-d513-44a7-9c09-36fd826deed6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.602353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 42/79] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:27.603192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bcdc7-7cda-4851-b950-a040d8897d1c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.603768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc34b17-173e-4a04-9d48-3da3e59122ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.605274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34b17-173e-4a04-9d48-3da3e59122ba None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.605561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34b17-173e-4a04-9d48-3da3e59122ba None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.605839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc34b17-173e-4a04-9d48-3da3e59122ba None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.607066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc34b17-173e-4a04-9d48-3da3e59122ba None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.608644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc34b17-173e-4a04-9d48-3da3e59122ba None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.610276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 38/80] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:27.612966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc34b17-173e-4a04-9d48-3da3e59122ba None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.701677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.704544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61bcdc7-7cda-4851-b950-a040d8897d1c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.711243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc34b17-173e-4a04-9d48-3da3e59122ba None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.733206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.738725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d10b-7cbe-42b3-93a0-1818f29ee79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.739524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d10b-7cbe-42b3-93a0-1818f29ee79b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.739811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d10b-7cbe-42b3-93a0-1818f29ee79b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.740094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0d10b-7cbe-42b3-93a0-1818f29ee79b None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.745790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf277c02-9492-42e5-bbf5-2ea27fa227b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.746593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf277c02-9492-42e5-bbf5-2ea27fa227b5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.746924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf277c02-9492-42e5-bbf5-2ea27fa227b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.747532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf277c02-9492-42e5-bbf5-2ea27fa227b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.749313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf277c02-9492-42e5-bbf5-2ea27fa227b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.750350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0d10b-7cbe-42b3-93a0-1818f29ee79b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:27.750952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf277c02-9492-42e5-bbf5-2ea27fa227b5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.752834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 39/81] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:27.753616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf277c02-9492-42e5-bbf5-2ea27fa227b5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-8ed0d10b-7cbe-42b3-93a0-1818f29ee79b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:27.763417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:27.764968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:27.765775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 43/82] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:27.767073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d10b-7cbe-42b3-93a0-1818f29ee79b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.855559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf277c02-9492-42e5-bbf5-2ea27fa227b5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.868706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0d10b-7cbe-42b3-93a0-1818f29ee79b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:27.890014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dc4fb-26ca-479a-ba8e-07738c5bfa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.890895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dc4fb-26ca-479a-ba8e-07738c5bfa59 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.891187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dc4fb-26ca-479a-ba8e-07738c5bfa59 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.891468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15dc4fb-26ca-479a-ba8e-07738c5bfa59 None admin] PATH_INFO: `/v3/users/2df7d0fe893f4cada8ecf1e428b8814a` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.892708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dc4fb-26ca-479a-ba8e-07738c5bfa59 None admin] RBAC: Authorizing `identity:get_user(user_id=2df7d0fe893f4cada8ecf1e428b8814a)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.894156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15dc4fb-26ca-479a-ba8e-07738c5bfa59 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.895796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 40/83] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/users/2df7d0fe893f4cada8ecf1e428b8814a => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:27.896595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dc4fb-26ca-479a-ba8e-07738c5bfa59 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.906493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd4406-27de-4a2e-81f5-14c2bf4e3bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:27.907657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd4406-27de-4a2e-81f5-14c2bf4e3bbe None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:27.907657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd4406-27de-4a2e-81f5-14c2bf4e3bbe None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:27.907900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd4406-27de-4a2e-81f5-14c2bf4e3bbe None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:27.909133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd4406-27de-4a2e-81f5-14c2bf4e3bbe None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:27.910594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd4406-27de-4a2e-81f5-14c2bf4e3bbe None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:27.912350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 44/84] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:27.913149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd4406-27de-4a2e-81f5-14c2bf4e3bbe None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:27.924986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.973136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:27.996132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15dc4fb-26ca-479a-ba8e-07738c5bfa59 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.015524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd4406-27de-4a2e-81f5-14c2bf4e3bbe None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.030350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eabd69-561a-4fd6-b4d2-e43e9d9722c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.031010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eabd69-561a-4fd6-b4d2-e43e9d9722c0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.031298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eabd69-561a-4fd6-b4d2-e43e9d9722c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.031585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eabd69-561a-4fd6-b4d2-e43e9d9722c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.048486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12120441-fc86-4b4b-aee6-07ae0d0b5cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.049170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12120441-fc86-4b4b-aee6-07ae0d0b5cf5 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.049457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12120441-fc86-4b4b-aee6-07ae0d0b5cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.049737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12120441-fc86-4b4b-aee6-07ae0d0b5cf5 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.050466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12120441-fc86-4b4b-aee6-07ae0d0b5cf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:28.051966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12120441-fc86-4b4b-aee6-07ae0d0b5cf5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.098650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eabd69-561a-4fd6-b4d2-e43e9d9722c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:28.100190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eabd69-561a-4fd6-b4d2-e43e9d9722c0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.126254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:28.130481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 45/85] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:27 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:28.131437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12120441-fc86-4b4b-aee6-07ae0d0b5cf5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.210548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 41/86] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:28.211433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eabd69-561a-4fd6-b4d2-e43e9d9722c0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.263827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12120441-fc86-4b4b-aee6-07ae0d0b5cf5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.280066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:28.305046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.305744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.306033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.306475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.307707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:28.308027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:28.309524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.320421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 46/87] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:28.321037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.328542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eabd69-561a-4fd6-b4d2-e43e9d9722c0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.363736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:28.376437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04365a0-5f94-45c3-b5a6-9debefc3b3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.377377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04365a0-5f94-45c3-b5a6-9debefc3b3bf None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.377840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04365a0-5f94-45c3-b5a6-9debefc3b3bf None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.378294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04365a0-5f94-45c3-b5a6-9debefc3b3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.445031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d57065-48ab-4392-8eab-ced728aa34eb None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.459406 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04365a0-5f94-45c3-b5a6-9debefc3b3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:28.460944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04365a0-5f94-45c3-b5a6-9debefc3b3bf None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.488674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.489336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.489618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.489896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.490738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] RBAC: Adding query filter params (role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd, user.id=388e066787d64c8f80e53b8373170ce1) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:28.491051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:28.492563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.502044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 47/88] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/role_assignments?user.id=388e066787d64c8f80e53b8373170ce1&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:28.503137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.611352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:28.681748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 42/89] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:28.684302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04365a0-5f94-45c3-b5a6-9debefc3b3bf None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.690400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c83f4-8fb4-440f-b972-efcb4f51e6b9 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.714919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:28.736514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afa3f3-6c53-435c-9f18-42c4c1e7f0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.737628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa3f3-6c53-435c-9f18-42c4c1e7f0b1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.738074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa3f3-6c53-435c-9f18-42c4c1e7f0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.738491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa3f3-6c53-435c-9f18-42c4c1e7f0b1 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.741085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa3f3-6c53-435c-9f18-42c4c1e7f0b1 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:28.742733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa3f3-6c53-435c-9f18-42c4c1e7f0b1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.746290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 48/90] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:28.748206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afa3f3-6c53-435c-9f18-42c4c1e7f0b1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.861230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04365a0-5f94-45c3-b5a6-9debefc3b3bf None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.928348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfee50-9ef9-4c2e-bca3-567752b85b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:28.929233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfee50-9ef9-4c2e-bca3-567752b85b7c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:28.929576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfee50-9ef9-4c2e-bca3-567752b85b7c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:28.929858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfee50-9ef9-4c2e-bca3-567752b85b7c None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:28.931161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfee50-9ef9-4c2e-bca3-567752b85b7c None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:28.933949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfee50-9ef9-4c2e-bca3-567752b85b7c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:28.935770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 43/91] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:28.941348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfee50-9ef9-4c2e-bca3-567752b85b7c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:28.946214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afa3f3-6c53-435c-9f18-42c4c1e7f0b1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:28.966590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.036823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275f651-f5eb-499f-b62f-e2d9c29a49e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.037767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275f651-f5eb-499f-b62f-e2d9c29a49e7 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.038105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275f651-f5eb-499f-b62f-e2d9c29a49e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.038395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275f651-f5eb-499f-b62f-e2d9c29a49e7 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.043819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275f651-f5eb-499f-b62f-e2d9c29a49e7 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.045498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275f651-f5eb-499f-b62f-e2d9c29a49e7 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.047468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 49/92] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:29.048062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275f651-f5eb-499f-b62f-e2d9c29a49e7 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.050516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.058306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfee50-9ef9-4c2e-bca3-567752b85b7c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.136413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.137155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.137451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.137734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.138590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] RBAC: Adding query filter params (role.id=20a552366bf14792b05decae218a79ee, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd, user.id=388e066787d64c8f80e53b8373170ce1) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:29.138909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.140466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.162106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 44/93] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:28 2022] GET /identity/v3/role_assignments?user.id=388e066787d64c8f80e53b8373170ce1&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=20a552366bf14792b05decae218a79ee => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:29.162964 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.189097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275f651-f5eb-499f-b62f-e2d9c29a49e7 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.234977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac89811-33fb-4b06-b298-2fecbbc5b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.236869 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac89811-33fb-4b06-b298-2fecbbc5b76f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.238482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac89811-33fb-4b06-b298-2fecbbc5b76f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.238980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac89811-33fb-4b06-b298-2fecbbc5b76f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.349812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.395979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac89811-33fb-4b06-b298-2fecbbc5b76f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.397623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac89811-33fb-4b06-b298-2fecbbc5b76f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.399131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f781c9-5961-4bb5-bc4d-333ba8f41171 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.447934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.449479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.449989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.450481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.451632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd, user.id=2df7d0fe893f4cada8ecf1e428b8814a) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:29.454441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.456284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.469721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 45/94] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/role_assignments?user.id=2df7d0fe893f4cada8ecf1e428b8814a&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 259 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:29.553063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 50/95] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:29.559431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.590522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.685297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5311a7-8658-40d6-ab27-de38e1ec8a03 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b8140-daeb-474e-a9f7-a3dac42255bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.732871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b8140-daeb-474e-a9f7-a3dac42255bd None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.733381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b8140-daeb-474e-a9f7-a3dac42255bd None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.733864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b8140-daeb-474e-a9f7-a3dac42255bd None admin] PATH_INFO: `/v3/projects/773944a297c545409a739c1982997f4e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.735376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b8140-daeb-474e-a9f7-a3dac42255bd None admin] RBAC: Authorizing `identity:get_project(project_id=773944a297c545409a739c1982997f4e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.737102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b8140-daeb-474e-a9f7-a3dac42255bd None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.739129 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 46/96] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/projects/773944a297c545409a739c1982997f4e => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:29.745561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac89811-33fb-4b06-b298-2fecbbc5b76f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:29.838271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:29.862899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac89811-33fb-4b06-b298-2fecbbc5b76f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:29.904493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c1fbbf-bd19-4a0b-ae3a-a5df38116313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:29.905164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1fbbf-bd19-4a0b-ae3a-a5df38116313 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:29.905450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1fbbf-bd19-4a0b-ae3a-a5df38116313 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:29.905728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c1fbbf-bd19-4a0b-ae3a-a5df38116313 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:29.906761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1fbbf-bd19-4a0b-ae3a-a5df38116313 None admin] RBAC: Adding query filter params (role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=773944a297c545409a739c1982997f4e, user.id=388e066787d64c8f80e53b8373170ce1) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:29.907081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1fbbf-bd19-4a0b-ae3a-a5df38116313 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:29.908581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c1fbbf-bd19-4a0b-ae3a-a5df38116313 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:29.914624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 51/97] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:29 2022] GET /identity/v3/role_assignments?user.id=388e066787d64c8f80e53b8373170ce1&scope.project.id=773944a297c545409a739c1982997f4e&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:31.059471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a421c-a3fc-4a72-b3fd-e6ef8e3de829 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.059973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a421c-a3fc-4a72-b3fd-e6ef8e3de829 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.059973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a421c-a3fc-4a72-b3fd-e6ef8e3de829 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.061467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 47/98] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:31.066667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7146c-0088-4182-9be9-7c6e54b996a0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.067069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7146c-0088-4182-9be9-7c6e54b996a0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.067225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7146c-0088-4182-9be9-7c6e54b996a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.090987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9fe7146c-0088-4182-9be9-7c6e54b996a0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:31.096352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9fe7146c-0088-4182-9be9-7c6e54b996a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:31.129350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146f5fc-064a-48a3-a256-ca742c7e0db0 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.129350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146f5fc-064a-48a3-a256-ca742c7e0db0 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.129350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146f5fc-064a-48a3-a256-ca742c7e0db0 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.129350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 48/99] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:31.134914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f34c-3d02-4db5-b09d-a3757ac5ed0e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.136312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f34c-3d02-4db5-b09d-a3757ac5ed0e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.136800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2758f34c-3d02-4db5-b09d-a3757ac5ed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.201182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2758f34c-3d02-4db5-b09d-a3757ac5ed0e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:31.202762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2758f34c-3d02-4db5-b09d-a3757ac5ed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:31.214228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 52/100] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:31.265294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7146c-0088-4182-9be9-7c6e54b996a0 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.342806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 49/101] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:31.378569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f34c-3d02-4db5-b09d-a3757ac5ed0e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.416528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.417562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.474018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7146c-0088-4182-9be9-7c6e54b996a0 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.535425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0544e4-acd1-465b-a780-13c77e40886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.536158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0544e4-acd1-465b-a780-13c77e40886e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.536451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0544e4-acd1-465b-a780-13c77e40886e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.536732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0544e4-acd1-465b-a780-13c77e40886e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.538721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0544e4-acd1-465b-a780-13c77e40886e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.540223 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0544e4-acd1-465b-a780-13c77e40886e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.542595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 53/102] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:31.543629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f170534-fcf6-412d-a03e-9749752c3c45 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.543915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f170534-fcf6-412d-a03e-9749752c3c45 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.544222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f170534-fcf6-412d-a03e-9749752c3c45 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.545463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 54/103] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:31.550432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f170534-fcf6-412d-a03e-9749752c3c45 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.564615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2758f34c-3d02-4db5-b09d-a3757ac5ed0e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.622318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a4b68-7bf1-45cc-be44-eb9288303820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.623056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4b68-7bf1-45cc-be44-eb9288303820 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.623342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4b68-7bf1-45cc-be44-eb9288303820 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.623624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4b68-7bf1-45cc-be44-eb9288303820 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.641218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a4b68-7bf1-45cc-be44-eb9288303820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-784a4b68-7bf1-45cc-be44-eb9288303820 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:31.648968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:31.649868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:31.650702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 50/104] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:31.651398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644a743-a779-4571-8be4-2631140bd734 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.651661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644a743-a779-4571-8be4-2631140bd734 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.655669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644a743-a779-4571-8be4-2631140bd734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.693752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d644a743-a779-4571-8be4-2631140bd734 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:31.696374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d644a743-a779-4571-8be4-2631140bd734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:31.735919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f170534-fcf6-412d-a03e-9749752c3c45 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:31.804999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad76f3-6ed4-4604-bd63-7cb43c11e4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:31.807390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad76f3-6ed4-4604-bd63-7cb43c11e4a9 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:31.807681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad76f3-6ed4-4604-bd63-7cb43c11e4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:31.808018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad76f3-6ed4-4604-bd63-7cb43c11e4a9 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:31.818459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad76f3-6ed4-4604-bd63-7cb43c11e4a9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:31.818459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad76f3-6ed4-4604-bd63-7cb43c11e4a9 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:31.829472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.839142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 51/105] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:31.840416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644a743-a779-4571-8be4-2631140bd734 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:31.872135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:31.921437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 55/106] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:40:31 2022] POST /identity/v3/projects => generated 302 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:31.923166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad76f3-6ed4-4604-bd63-7cb43c11e4a9 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.039048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644a743-a779-4571-8be4-2631140bd734 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.087446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.089476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.090013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.090487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.091454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] RBAC: Adding query filter params (name=reader) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:32.091970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.093803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.103576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 52/107] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.111392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.197793 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.219483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116cbf-8ab4-4c16-a1e4-32fe3c84eeea None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.226993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad76f3-6ed4-4604-bd63-7cb43c11e4a9 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.254981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f297e77-7f13-4b76-8f89-00b3fc7ca10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.255868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f297e77-7f13-4b76-8f89-00b3fc7ca10c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.256193 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f297e77-7f13-4b76-8f89-00b3fc7ca10c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.256477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f297e77-7f13-4b76-8f89-00b3fc7ca10c None admin] PATH_INFO: `/v3/users/7473f83f775a42ce87b115b180c8e94f` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.272272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f297e77-7f13-4b76-8f89-00b3fc7ca10c None admin] RBAC: Authorizing `identity:get_user(user_id=7473f83f775a42ce87b115b180c8e94f)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.273902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f297e77-7f13-4b76-8f89-00b3fc7ca10c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.275699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 53/108] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/users/7473f83f775a42ce87b115b180c8e94f => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:32.281762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f297e77-7f13-4b76-8f89-00b3fc7ca10c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.319050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca94e72-7bf6-465d-804f-8bb19636b394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.319868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94e72-7bf6-465d-804f-8bb19636b394 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.320183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94e72-7bf6-465d-804f-8bb19636b394 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.320471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94e72-7bf6-465d-804f-8bb19636b394 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.334025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca94e72-7bf6-465d-804f-8bb19636b394 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-4ca94e72-7bf6-465d-804f-8bb19636b394 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:32.342669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:32.343537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:32.344339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 56/109] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:32.345452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8bf6d-56bf-455a-8caf-c72c2e025089 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.345754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8bf6d-56bf-455a-8caf-c72c2e025089 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.346027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f8bf6d-56bf-455a-8caf-c72c2e025089 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.347438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 57/110] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:32.357415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8bf6d-56bf-455a-8caf-c72c2e025089 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.403085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f297e77-7f13-4b76-8f89-00b3fc7ca10c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.428579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.444149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.451302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6289d-47c5-4232-92ca-f8776c5c4565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.451894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6289d-47c5-4232-92ca-f8776c5c4565 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.452219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6289d-47c5-4232-92ca-f8776c5c4565 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.452504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6289d-47c5-4232-92ca-f8776c5c4565 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.491395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f8bf6d-56bf-455a-8caf-c72c2e025089 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.531213 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0af30-315d-4e3e-8f39-fad37392ea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.532252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0af30-315d-4e3e-8f39-fad37392ea53 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.532791 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0af30-315d-4e3e-8f39-fad37392ea53 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.534166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0af30-315d-4e3e-8f39-fad37392ea53 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.535132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0af30-315d-4e3e-8f39-fad37392ea53 None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:32.535628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0af30-315d-4e3e-8f39-fad37392ea53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.537432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0af30-315d-4e3e-8f39-fad37392ea53 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.554471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 58/111] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.555509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bfa95-8347-43fc-a54f-53af86372048 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.556963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bfa95-8347-43fc-a54f-53af86372048 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.558259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30bfa95-8347-43fc-a54f-53af86372048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.579211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6289d-47c5-4232-92ca-f8776c5c4565 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.580873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6289d-47c5-4232-92ca-f8776c5c4565 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.582476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d30bfa95-8347-43fc-a54f-53af86372048 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:32.585046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d30bfa95-8347-43fc-a54f-53af86372048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:32.667248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 59/112] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:32.668423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b6e7-b54f-49a8-912e-c60d4863f974 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.668768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b6e7-b54f-49a8-912e-c60d4863f974 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.669009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9b6e7-b54f-49a8-912e-c60d4863f974 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.670308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 60/113] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:32.671028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b6e7-b54f-49a8-912e-c60d4863f974 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.690011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 54/114] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:32.695778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1fe5-9f9e-4cd1-aef9-3076dbed82c6 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.696076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1fe5-9f9e-4cd1-aef9-3076dbed82c6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.696397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a1fe5-9f9e-4cd1-aef9-3076dbed82c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.715720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f08a1fe5-9f9e-4cd1-aef9-3076dbed82c6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:32.717223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f08a1fe5-9f9e-4cd1-aef9-3076dbed82c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:32.729862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.746894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.805721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9b6e7-b54f-49a8-912e-c60d4863f974 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.806076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 55/115] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:32.807538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a1fe5-9f9e-4cd1-aef9-3076dbed82c6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.845158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957948-17f8-43a2-90a8-4f5e722c93b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.846133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957948-17f8-43a2-90a8-4f5e722c93b0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.846489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957948-17f8-43a2-90a8-4f5e722c93b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.846836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11957948-17f8-43a2-90a8-4f5e722c93b0 None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.848860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11957948-17f8-43a2-90a8-4f5e722c93b0 None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.850627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11957948-17f8-43a2-90a8-4f5e722c93b0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.856226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 61/116] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.856226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957948-17f8-43a2-90a8-4f5e722c93b0 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.927501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a1fe5-9f9e-4cd1-aef9-3076dbed82c6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.964410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13b5e3-0b01-4888-85a5-4d70e9978100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:32.965285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b5e3-0b01-4888-85a5-4d70e9978100 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:32.965573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b5e3-0b01-4888-85a5-4d70e9978100 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:32.965913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13b5e3-0b01-4888-85a5-4d70e9978100 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:32.967066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13b5e3-0b01-4888-85a5-4d70e9978100 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:32.968560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13b5e3-0b01-4888-85a5-4d70e9978100 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:32.970281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 56/117] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:32.970281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13b5e3-0b01-4888-85a5-4d70e9978100 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:32.975155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11957948-17f8-43a2-90a8-4f5e722c93b0 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:32.985574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:32.999780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.015141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da0f1f-c878-4cb9-a126-c7f0f18eaf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.016344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0f1f-c878-4cb9-a126-c7f0f18eaf05 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.016780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0f1f-c878-4cb9-a126-c7f0f18eaf05 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.017209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0f1f-c878-4cb9-a126-c7f0f18eaf05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.029935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da0f1f-c878-4cb9-a126-c7f0f18eaf05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-74da0f1f-c878-4cb9-a126-c7f0f18eaf05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:33.038685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:33.039571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:33.040363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 62/118] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:33.041491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da0f1f-c878-4cb9-a126-c7f0f18eaf05 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.081780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13b5e3-0b01-4888-85a5-4d70e9978100 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.121344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e1fe8-4a70-4288-9233-1741732d08e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.122148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e1fe8-4a70-4288-9233-1741732d08e8 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.122440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e1fe8-4a70-4288-9233-1741732d08e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.123144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e1fe8-4a70-4288-9233-1741732d08e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.134313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e1fe8-4a70-4288-9233-1741732d08e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-0a3e1fe8-4a70-4288-9233-1741732d08e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:33.142559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:33.143400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:33.144210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 57/119] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:33.145482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42686a1a-2768-434f-8128-1bb25e2b4f52 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.145482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42686a1a-2768-434f-8128-1bb25e2b4f52 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.145621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42686a1a-2768-434f-8128-1bb25e2b4f52 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.146806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 58/120] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:33.152184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42686a1a-2768-434f-8128-1bb25e2b4f52 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.154467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da0f1f-c878-4cb9-a126-c7f0f18eaf05 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.189532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c570e-f472-4ce5-9c5e-17836b2a4bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.190409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c570e-f472-4ce5-9c5e-17836b2a4bef None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.190720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c570e-f472-4ce5-9c5e-17836b2a4bef None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.190973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c570e-f472-4ce5-9c5e-17836b2a4bef None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.192201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c570e-f472-4ce5-9c5e-17836b2a4bef None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.193661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c570e-f472-4ce5-9c5e-17836b2a4bef None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.200973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 63/121] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.200973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c570e-f472-4ce5-9c5e-17836b2a4bef None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.243385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.254091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.267894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42686a1a-2768-434f-8128-1bb25e2b4f52 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.303595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.304299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.304590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.304685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c570e-f472-4ce5-9c5e-17836b2a4bef None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.304874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.305719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] RBAC: Adding query filter params (role.id=f77ca21ddc4e463e98fbd3e9e7da7512, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd, user.id=7473f83f775a42ce87b115b180c8e94f) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:33.306044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.308377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.313871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 59/122] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/role_assignments?user.id=7473f83f775a42ce87b115b180c8e94f&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=f77ca21ddc4e463e98fbd3e9e7da7512 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.314676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.355626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062912b-63a0-4912-9523-bae50bb4371e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.356620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062912b-63a0-4912-9523-bae50bb4371e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.357088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062912b-63a0-4912-9523-bae50bb4371e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.357513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062912b-63a0-4912-9523-bae50bb4371e None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.358557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062912b-63a0-4912-9523-bae50bb4371e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:33.359032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062912b-63a0-4912-9523-bae50bb4371e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.360516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062912b-63a0-4912-9523-bae50bb4371e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.375115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 64/123] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.376684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6848-1ec4-48b7-84f4-ec0aa522f5d4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.377047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6848-1ec4-48b7-84f4-ec0aa522f5d4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.377340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6848-1ec4-48b7-84f4-ec0aa522f5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.395535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-57bf6848-1ec4-48b7-84f4-ec0aa522f5d4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:33.396803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-57bf6848-1ec4-48b7-84f4-ec0aa522f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:33.467181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d2ddc-3eba-4ecf-86a8-1ef919269c40 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.480079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 65/124] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:33.484288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf6848-1ec4-48b7-84f4-ec0aa522f5d4 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.527126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.530704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.532294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.532630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.532879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.533626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:33.538476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.538476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.552890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 60/125] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.553754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.575082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.608405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf6848-1ec4-48b7-84f4-ec0aa522f5d4 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.644656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471c396-3d99-4b11-a8f3-e2b9e5be92b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.645561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471c396-3d99-4b11-a8f3-e2b9e5be92b7 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.645850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471c396-3d99-4b11-a8f3-e2b9e5be92b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.646131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471c396-3d99-4b11-a8f3-e2b9e5be92b7 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/users/388e066787d64c8f80e53b8373170ce1/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.648232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471c396-3d99-4b11-a8f3-e2b9e5be92b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=39a8b4919bb540a7a257d26089c8b3bd, user_id=388e066787d64c8f80e53b8373170ce1, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.649714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471c396-3d99-4b11-a8f3-e2b9e5be92b7 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.679709 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb762c-79f0-4b6b-8fea-1f3f33c578c6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.719429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be0f56-6202-4c8f-a5a7-0af266db0606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.721470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be0f56-6202-4c8f-a5a7-0af266db0606 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.722121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be0f56-6202-4c8f-a5a7-0af266db0606 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.723089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be0f56-6202-4c8f-a5a7-0af266db0606 None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.732141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be0f56-6202-4c8f-a5a7-0af266db0606 None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:33.732141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be0f56-6202-4c8f-a5a7-0af266db0606 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:33.732141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 61/127] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:33.732141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be0f56-6202-4c8f-a5a7-0af266db0606 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.733079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 66/127] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:33 2022] PUT /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/users/388e066787d64c8f80e53b8373170ce1/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:33.734371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471c396-3d99-4b11-a8f3-e2b9e5be92b7 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:33.839835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.873980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be0f56-6202-4c8f-a5a7-0af266db0606 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.945750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:33.957676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471c396-3d99-4b11-a8f3-e2b9e5be92b7 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:33.971531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0c5a0-c682-4a6b-a76c-41dc3a881255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:33.972748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0c5a0-c682-4a6b-a76c-41dc3a881255 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:33.973230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0c5a0-c682-4a6b-a76c-41dc3a881255 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:33.973730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0c5a0-c682-4a6b-a76c-41dc3a881255 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:33.985434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0c5a0-c682-4a6b-a76c-41dc3a881255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-efb0c5a0-c682-4a6b-a76c-41dc3a881255 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:33.994520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:33.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 62/128] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:34.003373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0c5a0-c682-4a6b-a76c-41dc3a881255 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.004407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02137ccc-0ff1-486a-a7f9-5fc14a1e7268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.004407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02137ccc-0ff1-486a-a7f9-5fc14a1e7268 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.004407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02137ccc-0ff1-486a-a7f9-5fc14a1e7268 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.004407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02137ccc-0ff1-486a-a7f9-5fc14a1e7268 None admin] PATH_INFO: `/v3/users/2df7d0fe893f4cada8ecf1e428b8814a` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.007908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02137ccc-0ff1-486a-a7f9-5fc14a1e7268 None admin] RBAC: Authorizing `identity:get_user(user_id=2df7d0fe893f4cada8ecf1e428b8814a)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.011178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02137ccc-0ff1-486a-a7f9-5fc14a1e7268 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.013370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 67/129] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/users/2df7d0fe893f4cada8ecf1e428b8814a => generated 247 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:34.013842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02137ccc-0ff1-486a-a7f9-5fc14a1e7268 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.148257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0c5a0-c682-4a6b-a76c-41dc3a881255 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.174397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02137ccc-0ff1-486a-a7f9-5fc14a1e7268 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.184218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df53e6-5020-4d0a-bea0-a1e85a927d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.185102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df53e6-5020-4d0a-bea0-a1e85a927d58 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.185390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df53e6-5020-4d0a-bea0-a1e85a927d58 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.185673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df53e6-5020-4d0a-bea0-a1e85a927d58 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.186895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df53e6-5020-4d0a-bea0-a1e85a927d58 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.196141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df53e6-5020-4d0a-bea0-a1e85a927d58 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.196141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 63/130] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:33 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:34.196141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df53e6-5020-4d0a-bea0-a1e85a927d58 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.206254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.250863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.252603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.253137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.253607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.254564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:34.255066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.264210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.280317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 68/131] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:34.281543 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.315531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df53e6-5020-4d0a-bea0-a1e85a927d58 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.325048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.356653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73510ca1-95f5-4391-81a7-5b3ccb39bf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.358178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73510ca1-95f5-4391-81a7-5b3ccb39bf4c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.358756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73510ca1-95f5-4391-81a7-5b3ccb39bf4c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.359236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73510ca1-95f5-4391-81a7-5b3ccb39bf4c None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.360823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73510ca1-95f5-4391-81a7-5b3ccb39bf4c None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.362643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73510ca1-95f5-4391-81a7-5b3ccb39bf4c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.364606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 64/132] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:34.365630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73510ca1-95f5-4391-81a7-5b3ccb39bf4c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.470143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e258-c5a8-4ca4-a64f-c81d1db71869 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.472697 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.498235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73510ca1-95f5-4391-81a7-5b3ccb39bf4c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.506488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36a802-3fc6-4c96-b112-e72df19ae054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.507473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36a802-3fc6-4c96-b112-e72df19ae054 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.507835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36a802-3fc6-4c96-b112-e72df19ae054 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.508217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36a802-3fc6-4c96-b112-e72df19ae054 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/users/388e066787d64c8f80e53b8373170ce1/roles/20a552366bf14792b05decae218a79ee` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.511280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36a802-3fc6-4c96-b112-e72df19ae054 None admin] RBAC: Authorizing `identity:create_grant(project_id=39a8b4919bb540a7a257d26089c8b3bd, user_id=388e066787d64c8f80e53b8373170ce1, role_id=20a552366bf14792b05decae218a79ee)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.512880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36a802-3fc6-4c96-b112-e72df19ae054 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.540188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79e1d9-c4fd-4278-9d81-16d13645af2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.541243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79e1d9-c4fd-4278-9d81-16d13645af2f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.541530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79e1d9-c4fd-4278-9d81-16d13645af2f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.541810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79e1d9-c4fd-4278-9d81-16d13645af2f None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.543228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79e1d9-c4fd-4278-9d81-16d13645af2f None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.544798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca79e1d9-c4fd-4278-9d81-16d13645af2f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.546715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 65/133] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:34.552242 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79e1d9-c4fd-4278-9d81-16d13645af2f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.592029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 69/134] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:34 2022] PUT /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/users/388e066787d64c8f80e53b8373170ce1/roles/20a552366bf14792b05decae218a79ee => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:34.593665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf5f8c-dd8a-4d7e-98eb-067423da41f5 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.594109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf5f8c-dd8a-4d7e-98eb-067423da41f5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.594489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf5f8c-dd8a-4d7e-98eb-067423da41f5 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.601137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 70/135] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:34.602321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf5f8c-dd8a-4d7e-98eb-067423da41f5 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.722272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.742980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79e1d9-c4fd-4278-9d81-16d13645af2f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.748741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf5f8c-dd8a-4d7e-98eb-067423da41f5 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:34.796683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848f025-2420-4381-a81f-99d4de4f6078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.797740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848f025-2420-4381-a81f-99d4de4f6078 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.798139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848f025-2420-4381-a81f-99d4de4f6078 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.798523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848f025-2420-4381-a81f-99d4de4f6078 None admin] PATH_INFO: `/v3/projects/773944a297c545409a739c1982997f4e` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.799813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db664d0-1438-4ba2-a60d-a7afb743b6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:34.800164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848f025-2420-4381-a81f-99d4de4f6078 None admin] RBAC: Authorizing `identity:get_project(project_id=773944a297c545409a739c1982997f4e)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.800754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db664d0-1438-4ba2-a60d-a7afb743b6f9 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.801041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db664d0-1438-4ba2-a60d-a7afb743b6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.801323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db664d0-1438-4ba2-a60d-a7afb743b6f9 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/users/2df7d0fe893f4cada8ecf1e428b8814a/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.802039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848f025-2420-4381-a81f-99d4de4f6078 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.803376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db664d0-1438-4ba2-a60d-a7afb743b6f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=39a8b4919bb540a7a257d26089c8b3bd, user_id=2df7d0fe893f4cada8ecf1e428b8814a, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:34.804204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 71/136] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/projects/773944a297c545409a739c1982997f4e => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:34.805005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db664d0-1438-4ba2-a60d-a7afb743b6f9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:34.805560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc503d3-616e-4109-a3a9-e4f8dc109945 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:34.805911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc503d3-616e-4109-a3a9-e4f8dc109945 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:34.806280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc503d3-616e-4109-a3a9-e4f8dc109945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:34.825513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-5bc503d3-616e-4109-a3a9-e4f8dc109945 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:34.826801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5bc503d3-616e-4109-a3a9-e4f8dc109945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:34.906031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 72/137] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:34.906748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc503d3-616e-4109-a3a9-e4f8dc109945 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.931066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 66/138] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:34 2022] PUT /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/users/2df7d0fe893f4cada8ecf1e428b8814a/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:34.958117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db664d0-1438-4ba2-a60d-a7afb743b6f9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:34.978434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:34.999240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:35.024600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc503d3-616e-4109-a3a9-e4f8dc109945 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.063712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab13e-4542-4aad-b8ef-6af656fc4c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.064912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab13e-4542-4aad-b8ef-6af656fc4c4d None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.066316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab13e-4542-4aad-b8ef-6af656fc4c4d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.066792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ab13e-4542-4aad-b8ef-6af656fc4c4d None admin] PATH_INFO: `/v3/projects/773944a297c545409a739c1982997f4e/users/388e066787d64c8f80e53b8373170ce1/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.069129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ab13e-4542-4aad-b8ef-6af656fc4c4d None admin] RBAC: Authorizing `identity:create_grant(project_id=773944a297c545409a739c1982997f4e, user_id=388e066787d64c8f80e53b8373170ce1, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.071403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352ab13e-4542-4aad-b8ef-6af656fc4c4d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.159347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 73/139] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:34 2022] PUT /identity/v3/projects/773944a297c545409a739c1982997f4e/users/388e066787d64c8f80e53b8373170ce1/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:35.194279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db664d0-1438-4ba2-a60d-a7afb743b6f9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.239861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f773da25-4d3c-48e6-b829-5f4b8b2c1ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.240949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773da25-4d3c-48e6-b829-5f4b8b2c1ffd None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.241384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773da25-4d3c-48e6-b829-5f4b8b2c1ffd None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.241806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f773da25-4d3c-48e6-b829-5f4b8b2c1ffd None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.243278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773da25-4d3c-48e6-b829-5f4b8b2c1ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.245585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f773da25-4d3c-48e6-b829-5f4b8b2c1ffd None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.247470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 67/140] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:35.253309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab13e-4542-4aad-b8ef-6af656fc4c4d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:35.366348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352ab13e-4542-4aad-b8ef-6af656fc4c4d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:35.404963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc1948-e2f6-41c3-bf05-615581209a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:35.405916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1948-e2f6-41c3-bf05-615581209a54 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.406339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1948-e2f6-41c3-bf05-615581209a54 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.406758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1948-e2f6-41c3-bf05-615581209a54 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.407633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc1948-e2f6-41c3-bf05-615581209a54 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:35.409281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc1948-e2f6-41c3-bf05-615581209a54 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:35.419761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:35.532075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 74/141] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:35 2022] POST /identity/v3/users => generated 283 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:35.861163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc47f38-6732-4c72-bb8d-28cb31449ba1 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.861466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc47f38-6732-4c72-bb8d-28cb31449ba1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.861660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc47f38-6732-4c72-bb8d-28cb31449ba1 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.863064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 68/142] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:35.868665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f39c-623b-4ce6-82e4-3507e1dd6472 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:35.868953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f39c-623b-4ce6-82e4-3507e1dd6472 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:35.869259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3f39c-623b-4ce6-82e4-3507e1dd6472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:35.896247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-bbd3f39c-623b-4ce6-82e4-3507e1dd6472 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:35.897439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bbd3f39c-623b-4ce6-82e4-3507e1dd6472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:35.988277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 75/143] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:36.014738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc47f38-6732-4c72-bb8d-28cb31449ba1 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.100845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.128757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc47f38-6732-4c72-bb8d-28cb31449ba1 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.166986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d13ff-ba4a-4c36-8168-fe75b99d3ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.167862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d13ff-ba4a-4c36-8168-fe75b99d3ab0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.168167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d13ff-ba4a-4c36-8168-fe75b99d3ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.168453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d13ff-ba4a-4c36-8168-fe75b99d3ab0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.179145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d13ff-ba4a-4c36-8168-fe75b99d3ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-5a8d13ff-ba4a-4c36-8168-fe75b99d3ab0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:36.187588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:36.188530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:36.189296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 69/144] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:36.196393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3f39c-623b-4ce6-82e4-3507e1dd6472 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.324616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3f39c-623b-4ce6-82e4-3507e1dd6472 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.350818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.364318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.365040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.365321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.365628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.366351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:36.366649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.368065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.378826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 76/145] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:36.384860 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d13ff-ba4a-4c36-8168-fe75b99d3ab0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.570414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d13ff-ba4a-4c36-8168-fe75b99d3ab0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.610155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c093c3-e431-4354-aeac-b035af4d5464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.611102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c093c3-e431-4354-aeac-b035af4d5464 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.611389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c093c3-e431-4354-aeac-b035af4d5464 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.611666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c093c3-e431-4354-aeac-b035af4d5464 None admin] PATH_INFO: `/v3/users/7473f83f775a42ce87b115b180c8e94f` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.612961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c093c3-e431-4354-aeac-b035af4d5464 None admin] RBAC: Authorizing `identity:get_user(user_id=7473f83f775a42ce87b115b180c8e94f)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.614442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c093c3-e431-4354-aeac-b035af4d5464 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.620571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 70/146] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/users/7473f83f775a42ce87b115b180c8e94f => generated 289 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:36.622611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.788596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d0994-dd3f-47cc-bca0-9a64ccf40fc5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:36.829322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa66719-90ef-41b5-8479-ad942d61956a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:36.830194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa66719-90ef-41b5-8479-ad942d61956a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:36.830477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa66719-90ef-41b5-8479-ad942d61956a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:36.830751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa66719-90ef-41b5-8479-ad942d61956a None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:36.831979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa66719-90ef-41b5-8479-ad942d61956a None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:36.833611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa66719-90ef-41b5-8479-ad942d61956a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:36.835320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 77/147] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:36 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:36.841702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c093c3-e431-4354-aeac-b035af4d5464 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:36.844680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:36.964155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c093c3-e431-4354-aeac-b035af4d5464 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.000535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09318b3b-c3ff-4a58-bbbb-847234614d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.001434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09318b3b-c3ff-4a58-bbbb-847234614d58 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.001721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09318b3b-c3ff-4a58-bbbb-847234614d58 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.002000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09318b3b-c3ff-4a58-bbbb-847234614d58 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/users/7473f83f775a42ce87b115b180c8e94f/roles/f77ca21ddc4e463e98fbd3e9e7da7512` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.004007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09318b3b-c3ff-4a58-bbbb-847234614d58 None admin] RBAC: Authorizing `identity:create_grant(project_id=39a8b4919bb540a7a257d26089c8b3bd, user_id=7473f83f775a42ce87b115b180c8e94f, role_id=f77ca21ddc4e463e98fbd3e9e7da7512)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.005504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09318b3b-c3ff-4a58-bbbb-847234614d58 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.080788 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 71/148] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:36 2022] PUT /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/users/7473f83f775a42ce87b115b180c8e94f/roles/f77ca21ddc4e463e98fbd3e9e7da7512 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:37.144798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05da85-5cea-4c41-a626-989e9e88abad None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.145095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05da85-5cea-4c41-a626-989e9e88abad None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.145370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05da85-5cea-4c41-a626-989e9e88abad None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.147127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 78/149] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:37.153462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf26947-7114-498f-b025-d5aadbe58a8d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.153766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf26947-7114-498f-b025-d5aadbe58a8d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.154038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf26947-7114-498f-b025-d5aadbe58a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.177369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7bf26947-7114-498f-b025-d5aadbe58a8d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:37.178610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7bf26947-7114-498f-b025-d5aadbe58a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:37.255039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.269616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 72/150] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:37.329278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05da85-5cea-4c41-a626-989e9e88abad None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.393429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.441393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05da85-5cea-4c41-a626-989e9e88abad None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.483417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902ee64-4a65-4e86-b24b-0795ea62bf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.484253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902ee64-4a65-4e86-b24b-0795ea62bf49 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.484536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902ee64-4a65-4e86-b24b-0795ea62bf49 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.484807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5902ee64-4a65-4e86-b24b-0795ea62bf49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.501621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5902ee64-4a65-4e86-b24b-0795ea62bf49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-5902ee64-4a65-4e86-b24b-0795ea62bf49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:37.510383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:37.511335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:37.512156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:37.514372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 79/151] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:37.519496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf26947-7114-498f-b025-d5aadbe58a8d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.632170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf26947-7114-498f-b025-d5aadbe58a8d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.671934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.672705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.672932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.673215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.674045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] RBAC: Adding query filter params (name=member) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:37.674363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.675811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.687331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 73/152] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:37.691248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902ee64-4a65-4e86-b24b-0795ea62bf49 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.826841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5902ee64-4a65-4e86-b24b-0795ea62bf49 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:37.844631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:37.868154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fca132-e8af-4de8-9288-e7dd487c7d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:37.869576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fca132-e8af-4de8-9288-e7dd487c7d67 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:37.869871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fca132-e8af-4de8-9288-e7dd487c7d67 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:37.870140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fca132-e8af-4de8-9288-e7dd487c7d67 None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:37.871375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fca132-e8af-4de8-9288-e7dd487c7d67 None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:37.874135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fca132-e8af-4de8-9288-e7dd487c7d67 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:37.875831 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 80/153] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:37.881701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:37.959554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.002633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf00a5-5b8e-4f69-b368-79704dd846cf None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.003329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf00a5-5b8e-4f69-b368-79704dd846cf None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.003823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf00a5-5b8e-4f69-b368-79704dd846cf None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.006098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 81/154] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:38.011142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d8442-8063-4e2e-b62e-68f6b621463b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.011753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d8442-8063-4e2e-b62e-68f6b621463b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.012260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d8442-8063-4e2e-b62e-68f6b621463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.016302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3379d3-6782-4fe7-92fd-1c6b79b8fbee None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.043585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4b7d8442-8063-4e2e-b62e-68f6b621463b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:38.045282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4b7d8442-8063-4e2e-b62e-68f6b621463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:38.059754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9353c7-6372-497c-a5e2-f858d766bb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.060449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9353c7-6372-497c-a5e2-f858d766bb2e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.060759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9353c7-6372-497c-a5e2-f858d766bb2e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.064217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9353c7-6372-497c-a5e2-f858d766bb2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.154820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9353c7-6372-497c-a5e2-f858d766bb2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.160403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9353c7-6372-497c-a5e2-f858d766bb2e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.187087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 82/155] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:38.232231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.239475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d8442-8063-4e2e-b62e-68f6b621463b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.301591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 74/156] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:38.314308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9353c7-6372-497c-a5e2-f858d766bb2e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.439316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d8442-8063-4e2e-b62e-68f6b621463b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.461266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.500551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c630328-afa0-4520-9dbc-5a4930bdedba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.501304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c630328-afa0-4520-9dbc-5a4930bdedba None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.501592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c630328-afa0-4520-9dbc-5a4930bdedba None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.502110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c630328-afa0-4520-9dbc-5a4930bdedba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.517340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9353c7-6372-497c-a5e2-f858d766bb2e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.526981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c630328-afa0-4520-9dbc-5a4930bdedba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-6c630328-afa0-4520-9dbc-5a4930bdedba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:38.535955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:38.536891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:38.537672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 83/157] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:38.545330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c630328-afa0-4520-9dbc-5a4930bdedba None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.588837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253ecd-355e-48f7-abe6-12bb99eded76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.590545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253ecd-355e-48f7-abe6-12bb99eded76 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.591047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253ecd-355e-48f7-abe6-12bb99eded76 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.591584 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253ecd-355e-48f7-abe6-12bb99eded76 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.593196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b253ecd-355e-48f7-abe6-12bb99eded76 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.594999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b253ecd-355e-48f7-abe6-12bb99eded76 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.597366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 75/158] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:38.604102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253ecd-355e-48f7-abe6-12bb99eded76 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.643114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.688850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c630328-afa0-4520-9dbc-5a4930bdedba None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.746531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bef0ee-aca0-48c4-ab03-d67efb3f79ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.747346 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bef0ee-aca0-48c4-ab03-d67efb3f79ec None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.747639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bef0ee-aca0-48c4-ab03-d67efb3f79ec None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.747918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bef0ee-aca0-48c4-ab03-d67efb3f79ec None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.748715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bef0ee-aca0-48c4-ab03-d67efb3f79ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.749117 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bef0ee-aca0-48c4-ab03-d67efb3f79ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.752624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bef0ee-aca0-48c4-ab03-d67efb3f79ec None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.775829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 84/159] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:38.775829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7c1e1-a989-48bd-bc8f-071cf9656086 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.775829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7c1e1-a989-48bd-bc8f-071cf9656086 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.775829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7c1e1-a989-48bd-bc8f-071cf9656086 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.775829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 85/160] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:38.777124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7c1e1-a989-48bd-bc8f-071cf9656086 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:38.809135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253ecd-355e-48f7-abe6-12bb99eded76 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:38.826858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:38.892272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db4271-2015-4a8a-bab0-0f95b3acc81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:38.893051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db4271-2015-4a8a-bab0-0f95b3acc81b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.893396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db4271-2015-4a8a-bab0-0f95b3acc81b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.893678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db4271-2015-4a8a-bab0-0f95b3acc81b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.894526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db4271-2015-4a8a-bab0-0f95b3acc81b None admin] RBAC: Adding query filter params (role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd, user.id=388e066787d64c8f80e53b8373170ce1) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:38.894938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db4271-2015-4a8a-bab0-0f95b3acc81b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:38.896433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db4271-2015-4a8a-bab0-0f95b3acc81b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:38.907009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 76/161] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/role_assignments?user.id=388e066787d64c8f80e53b8373170ce1&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:38.908255 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814950a8-f9b9-42b8-916d-f59473e1b341 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:38.908447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814950a8-f9b9-42b8-916d-f59473e1b341 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:38.908723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814950a8-f9b9-42b8-916d-f59473e1b341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:38.945158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-814950a8-f9b9-42b8-916d-f59473e1b341 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:38.946731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-814950a8-f9b9-42b8-916d-f59473e1b341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:38.947074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7c1e1-a989-48bd-bc8f-071cf9656086 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.003907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb587f4-3816-4d1c-9050-9b4fb3b12a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.004764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb587f4-3816-4d1c-9050-9b4fb3b12a42 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.005046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb587f4-3816-4d1c-9050-9b4fb3b12a42 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.005323 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb587f4-3816-4d1c-9050-9b4fb3b12a42 None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.007732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb587f4-3816-4d1c-9050-9b4fb3b12a42 None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.010216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb587f4-3816-4d1c-9050-9b4fb3b12a42 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.012885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 86/162] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:38 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:39.021793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb587f4-3816-4d1c-9050-9b4fb3b12a42 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.109425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 77/163] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:39.111011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4fda2-5ac7-4635-a73a-ba97c80257d1 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.111463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4fda2-5ac7-4635-a73a-ba97c80257d1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.111939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4fda2-5ac7-4635-a73a-ba97c80257d1 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.113387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 78/164] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:39.131958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655fad-80c2-4f22-8280-9c5adff50b66 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.132622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655fad-80c2-4f22-8280-9c5adff50b66 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.133101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae655fad-80c2-4f22-8280-9c5adff50b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.179412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ae655fad-80c2-4f22-8280-9c5adff50b66 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:39.181056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ae655fad-80c2-4f22-8280-9c5adff50b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:39.207523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.223393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb587f4-3816-4d1c-9050-9b4fb3b12a42 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.232531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.265825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1a88c-0207-4cb6-bd87-7be462113f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.266393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1a88c-0207-4cb6-bd87-7be462113f72 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.266734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1a88c-0207-4cb6-bd87-7be462113f72 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.267017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1a88c-0207-4cb6-bd87-7be462113f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.330207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 79/165] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:39.331412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae655fad-80c2-4f22-8280-9c5adff50b66 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.350411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1a88c-0207-4cb6-bd87-7be462113f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.352796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1a88c-0207-4cb6-bd87-7be462113f72 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.482768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 87/166] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:39.488229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1a88c-0207-4cb6-bd87-7be462113f72 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.491529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.560841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae655fad-80c2-4f22-8280-9c5adff50b66 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.600088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1a88c-0207-4cb6-bd87-7be462113f72 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.611804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400e145-80cd-4474-bbda-31d49e995f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.612661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400e145-80cd-4474-bbda-31d49e995f98 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.612949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400e145-80cd-4474-bbda-31d49e995f98 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.613230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400e145-80cd-4474-bbda-31d49e995f98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.629131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e400e145-80cd-4474-bbda-31d49e995f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-e400e145-80cd-4474-bbda-31d49e995f98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:39.638563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:39.639509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:39.640518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 80/167] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:39.642690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400e145-80cd-4474-bbda-31d49e995f98 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.645787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e8a0d-8fe1-4cee-9a98-d8228f22c62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.646607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e8a0d-8fe1-4cee-9a98-d8228f22c62f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.646889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e8a0d-8fe1-4cee-9a98-d8228f22c62f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.647160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e8a0d-8fe1-4cee-9a98-d8228f22c62f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.668141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.673422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e8a0d-8fe1-4cee-9a98-d8228f22c62f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-e15e8a0d-8fe1-4cee-9a98-d8228f22c62f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:39.683756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:39.685978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:39.687936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 88/168] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:39.696106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e8a0d-8fe1-4cee-9a98-d8228f22c62f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.838589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400e145-80cd-4474-bbda-31d49e995f98 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:39.849772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:39.907397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861df1b-a3cb-44c4-81f7-590fc1894aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:39.908408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861df1b-a3cb-44c4-81f7-590fc1894aef None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:39.908696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861df1b-a3cb-44c4-81f7-590fc1894aef None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:39.908977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861df1b-a3cb-44c4-81f7-590fc1894aef None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:39.910239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861df1b-a3cb-44c4-81f7-590fc1894aef None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:39.911730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861df1b-a3cb-44c4-81f7-590fc1894aef None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:39.916504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 81/169] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:39.917526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861df1b-a3cb-44c4-81f7-590fc1894aef None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:39.948147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e8a0d-8fe1-4cee-9a98-d8228f22c62f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.026023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.026726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.027017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.027297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.028065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:40.028400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.029796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.047361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 89/170] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.048011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.048322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.122517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861df1b-a3cb-44c4-81f7-590fc1894aef None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.196035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.197194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.197825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.199249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.200821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] RBAC: Adding query filter params (name=member) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:40.201599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.203197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.213533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 82/171] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.215429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.310340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1373d7-a42c-4cf3-812e-785b034211dc None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.332236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935666f3-1417-4fb8-9634-7e093f7b2824 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.369815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce73d2d-6dd7-4cef-b78d-6dfc37e08bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.371614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73d2d-6dd7-4cef-b78d-6dfc37e08bad None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.372235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73d2d-6dd7-4cef-b78d-6dfc37e08bad None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.373230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce73d2d-6dd7-4cef-b78d-6dfc37e08bad None admin] PATH_INFO: `/v3/users/2df7d0fe893f4cada8ecf1e428b8814a` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.375152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce73d2d-6dd7-4cef-b78d-6dfc37e08bad None admin] RBAC: Authorizing `identity:get_user(user_id=2df7d0fe893f4cada8ecf1e428b8814a)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.377053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce73d2d-6dd7-4cef-b78d-6dfc37e08bad None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.378938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 83/172] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/users/2df7d0fe893f4cada8ecf1e428b8814a => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.380813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0db32d-a9a0-4469-b3ed-f0d1a12e10dc None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.381329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0db32d-a9a0-4469-b3ed-f0d1a12e10dc None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.382140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.382964 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0db32d-a9a0-4469-b3ed-f0d1a12e10dc None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.384625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 84/173] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:40.385842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0db32d-a9a0-4469-b3ed-f0d1a12e10dc None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.401051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.438236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.438912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.439199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.439476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.440374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] RBAC: Adding query filter params (role.id=20a552366bf14792b05decae218a79ee, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd, user.id=388e066787d64c8f80e53b8373170ce1) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:40.440696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.442159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.461300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 90/174] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/role_assignments?user.id=388e066787d64c8f80e53b8373170ce1&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=20a552366bf14792b05decae218a79ee => generated 608 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.462322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.596237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0db32d-a9a0-4469-b3ed-f0d1a12e10dc None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.633913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f3a12-9def-4cde-8856-80df7a7c38b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.635381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f3a12-9def-4cde-8856-80df7a7c38b1 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.635572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f3a12-9def-4cde-8856-80df7a7c38b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.635825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f3a12-9def-4cde-8856-80df7a7c38b1 None admin] PATH_INFO: `/v3/users/388e066787d64c8f80e53b8373170ce1` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.637325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f3a12-9def-4cde-8856-80df7a7c38b1 None admin] RBAC: Authorizing `identity:get_user(user_id=388e066787d64c8f80e53b8373170ce1)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.638803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f3a12-9def-4cde-8856-80df7a7c38b1 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.640558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 85/175] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/users/388e066787d64c8f80e53b8373170ce1 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:40.645570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf65da-2ee9-459b-a39c-84a59dfe86fd None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.645908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf65da-2ee9-459b-a39c-84a59dfe86fd None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.646133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf65da-2ee9-459b-a39c-84a59dfe86fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.668786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ddbf65da-2ee9-459b-a39c-84a59dfe86fd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:40.670025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ddbf65da-2ee9-459b-a39c-84a59dfe86fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:40.712487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f0671-2dae-414d-88ba-d00b028b9cef None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.759174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e25698-82f1-4f2f-b7ec-5093478e6769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.760409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e25698-82f1-4f2f-b7ec-5093478e6769 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.760688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e25698-82f1-4f2f-b7ec-5093478e6769 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.760972 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e25698-82f1-4f2f-b7ec-5093478e6769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.793089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 86/176] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:40.801788 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170879-d07b-48cc-b69b-0c4a5c37e12a None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.802241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170879-d07b-48cc-b69b-0c4a5c37e12a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.802664 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17170879-d07b-48cc-b69b-0c4a5c37e12a None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.804278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 87/177] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:40.805297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17170879-d07b-48cc-b69b-0c4a5c37e12a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:40.811098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.837743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:40.853949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e25698-82f1-4f2f-b7ec-5093478e6769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:40.856310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e25698-82f1-4f2f-b7ec-5093478e6769 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:40.948940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17170879-d07b-48cc-b69b-0c4a5c37e12a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:40.992578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221d918-7d14-4c5c-b6ca-e330590219eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:40.993256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d918-7d14-4c5c-b6ca-e330590219eb None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.993523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d918-7d14-4c5c-b6ca-e330590219eb None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.993804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221d918-7d14-4c5c-b6ca-e330590219eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:40.994532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 91/178] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:40.995718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6772317-03a2-4862-82a5-d9d89730e849 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:40.996006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6772317-03a2-4862-82a5-d9d89730e849 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:40.996311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6772317-03a2-4862-82a5-d9d89730e849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.011970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d6772317-03a2-4862-82a5-d9d89730e849 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:41.013203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d6772317-03a2-4862-82a5-d9d89730e849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:41.061882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221d918-7d14-4c5c-b6ca-e330590219eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.063416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7221d918-7d14-4c5c-b6ca-e330590219eb None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.065267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.080182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 92/179] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:41.081068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6772317-03a2-4862-82a5-d9d89730e849 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.106623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.164351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 88/180] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:41.165522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532b47-2b65-4ca6-a479-511e34a6de78 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.166540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532b47-2b65-4ca6-a479-511e34a6de78 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.167051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db532b47-2b65-4ca6-a479-511e34a6de78 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.168331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 89/181] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:41.172600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9fcdc-06b5-448f-a9bf-2217a38acfd7 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.172890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9fcdc-06b5-448f-a9bf-2217a38acfd7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.173171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf9fcdc-06b5-448f-a9bf-2217a38acfd7 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.174434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 90/182] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:41.176868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9fcdc-06b5-448f-a9bf-2217a38acfd7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.190737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6772317-03a2-4862-82a5-d9d89730e849 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.225626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a94c6c-f53b-4455-b2f2-85237da9c88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.226451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94c6c-f53b-4455-b2f2-85237da9c88e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.226737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94c6c-f53b-4455-b2f2-85237da9c88e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.227012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94c6c-f53b-4455-b2f2-85237da9c88e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.237837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a94c6c-f53b-4455-b2f2-85237da9c88e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-80a94c6c-f53b-4455-b2f2-85237da9c88e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:41.245549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:41.246532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:41.247319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 93/183] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:41.247968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a94c6c-f53b-4455-b2f2-85237da9c88e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.283673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf9fcdc-06b5-448f-a9bf-2217a38acfd7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.285241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.317054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f537ff-ff3e-4f3f-9567-e10bcfc97cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.317909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f537ff-ff3e-4f3f-9567-e10bcfc97cfb None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.318190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f537ff-ff3e-4f3f-9567-e10bcfc97cfb None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.318468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f537ff-ff3e-4f3f-9567-e10bcfc97cfb None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.319634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f537ff-ff3e-4f3f-9567-e10bcfc97cfb None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.321107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f537ff-ff3e-4f3f-9567-e10bcfc97cfb None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.323729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 91/184] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:41.324519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f537ff-ff3e-4f3f-9567-e10bcfc97cfb None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.324960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.346769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a94c6c-f53b-4455-b2f2-85237da9c88e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.380227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26441339-2cfe-4210-84a3-481193ab19b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.381067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26441339-2cfe-4210-84a3-481193ab19b8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.381353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26441339-2cfe-4210-84a3-481193ab19b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.381629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26441339-2cfe-4210-84a3-481193ab19b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.392030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26441339-2cfe-4210-84a3-481193ab19b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-26441339-2cfe-4210-84a3-481193ab19b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:41.400794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:41.402071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:41.402850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 94/185] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:41.403490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12d3b9-4e02-4f0d-9945-5dbd681576aa None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.403490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12d3b9-4e02-4f0d-9945-5dbd681576aa None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.403619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12d3b9-4e02-4f0d-9945-5dbd681576aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.419376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fb12d3b9-4e02-4f0d-9945-5dbd681576aa None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:41.420616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fb12d3b9-4e02-4f0d-9945-5dbd681576aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:41.427336 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f537ff-ff3e-4f3f-9567-e10bcfc97cfb None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.461299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab39e6-97b6-4fef-9567-c6ce6c95a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.462167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab39e6-97b6-4fef-9567-c6ce6c95a836 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.462450 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab39e6-97b6-4fef-9567-c6ce6c95a836 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.463353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab39e6-97b6-4fef-9567-c6ce6c95a836 None admin] PATH_INFO: `/v3/projects/773944a297c545409a739c1982997f4e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.464600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab39e6-97b6-4fef-9567-c6ce6c95a836 None admin] RBAC: Authorizing `identity:get_project(project_id=773944a297c545409a739c1982997f4e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.466054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab39e6-97b6-4fef-9567-c6ce6c95a836 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.467760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 92/186] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/projects/773944a297c545409a739c1982997f4e => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:41.468921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad43e5e-386e-4524-a9d6-ffde6b81421b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.469212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad43e5e-386e-4524-a9d6-ffde6b81421b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.469492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad43e5e-386e-4524-a9d6-ffde6b81421b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.485124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-aad43e5e-386e-4524-a9d6-ffde6b81421b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:41.486241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-aad43e5e-386e-4524-a9d6-ffde6b81421b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:41.492479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 95/187] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:41.493323 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12d3b9-4e02-4f0d-9945-5dbd681576aa None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.519618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.568143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 93/188] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:41.568143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad43e5e-386e-4524-a9d6-ffde6b81421b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.580749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.596432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12d3b9-4e02-4f0d-9945-5dbd681576aa None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.631466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.632167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.632466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.632745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.633498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:41.633807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.635904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.644005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 96/189] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:41.644669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.657536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad43e5e-386e-4524-a9d6-ffde6b81421b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.691267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.691922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.692237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.692520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.693363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd, user.id=2df7d0fe893f4cada8ecf1e428b8814a) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:41.693679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.695128 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.701729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 94/190] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/role_assignments?user.id=2df7d0fe893f4cada8ecf1e428b8814a&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:41.701729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.737838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.769105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77c381-befc-4a45-b005-0aa3ce109d6c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.801273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:41.808831 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.809531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.809819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.810096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.813887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:41.813887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.813887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.822745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 97/191] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:41.823666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.834805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a332bb8-5f56-48f1-8316-f398af0d8715 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.878106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.878916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.879304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.879706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.880732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] RBAC: Adding query filter params (role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=773944a297c545409a739c1982997f4e, user.id=388e066787d64c8f80e53b8373170ce1) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:41.881128 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:41.882689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:41.890755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 95/192] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/role_assignments?user.id=388e066787d64c8f80e53b8373170ce1&scope.project.id=773944a297c545409a739c1982997f4e&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:41.891545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:41.956363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe43c0-b69c-40ab-b733-43fb0e23fd46 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:41.995989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d18dd-d8d0-492c-a94c-4058ae604fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:41.997199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d18dd-d8d0-492c-a94c-4058ae604fb3 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:41.997623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d18dd-d8d0-492c-a94c-4058ae604fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:41.998054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d18dd-d8d0-492c-a94c-4058ae604fb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:41.999589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d18dd-d8d0-492c-a94c-4058ae604fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.001363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d18dd-d8d0-492c-a94c-4058ae604fb3 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.003399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 98/193] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:42.004417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d18dd-d8d0-492c-a94c-4058ae604fb3 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.023882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.030000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7afef5-cdc5-4451-998e-1106bf3bee1a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.074169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a04e-321f-4ce7-b256-0beb6c0ccad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.075112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a04e-321f-4ce7-b256-0beb6c0ccad7 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.075395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a04e-321f-4ce7-b256-0beb6c0ccad7 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.075669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71a04e-321f-4ce7-b256-0beb6c0ccad7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.075918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.087189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71a04e-321f-4ce7-b256-0beb6c0ccad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-6b71a04e-321f-4ce7-b256-0beb6c0ccad7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:42.095636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:42.096809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:42.097763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 96/194] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:42.098560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a04e-321f-4ce7-b256-0beb6c0ccad7 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.129659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d18dd-d8d0-492c-a94c-4058ae604fb3 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.164548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d607f82-9417-4282-8429-a6f017acf486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.165430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d607f82-9417-4282-8429-a6f017acf486 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.165787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d607f82-9417-4282-8429-a6f017acf486 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.166082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d607f82-9417-4282-8429-a6f017acf486 None admin] PATH_INFO: `/v3/users/0f1cb372d7a641fc9948b38a438106aa` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.179149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d607f82-9417-4282-8429-a6f017acf486 None admin] RBAC: Authorizing `identity:get_user(user_id=0f1cb372d7a641fc9948b38a438106aa)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.180641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d607f82-9417-4282-8429-a6f017acf486 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.182444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 99/195] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/users/0f1cb372d7a641fc9948b38a438106aa => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:42.187192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d607f82-9417-4282-8429-a6f017acf486 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.202610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71a04e-321f-4ce7-b256-0beb6c0ccad7 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.235922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8208a1-3f63-4159-baba-959f1c7a74cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.236836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8208a1-3f63-4159-baba-959f1c7a74cd None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.237125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8208a1-3f63-4159-baba-959f1c7a74cd None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.237403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8208a1-3f63-4159-baba-959f1c7a74cd None admin] PATH_INFO: `/v3/users/0f1cb372d7a641fc9948b38a438106aa` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.238606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8208a1-3f63-4159-baba-959f1c7a74cd None admin] RBAC: Authorizing `identity:get_user(user_id=0f1cb372d7a641fc9948b38a438106aa)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.240058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8208a1-3f63-4159-baba-959f1c7a74cd None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.241724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 97/196] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/users/0f1cb372d7a641fc9948b38a438106aa => generated 283 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:42.245086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8208a1-3f63-4159-baba-959f1c7a74cd None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.270290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.285532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d607f82-9417-4282-8429-a6f017acf486 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.302534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.318623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f85b3c-1c92-4a0c-aaeb-7a80f5bd182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.319296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f85b3c-1c92-4a0c-aaeb-7a80f5bd182e None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.319591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f85b3c-1c92-4a0c-aaeb-7a80f5bd182e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.319870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f85b3c-1c92-4a0c-aaeb-7a80f5bd182e None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.320615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f85b3c-1c92-4a0c-aaeb-7a80f5bd182e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.322661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f85b3c-1c92-4a0c-aaeb-7a80f5bd182e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.345282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8208a1-3f63-4159-baba-959f1c7a74cd None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.378854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.379515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.379801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.380077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.380859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:42.381194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.382638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.391461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 98/197] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:42.392313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.407795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 100/198] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:42 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:42.407795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f85b3c-1c92-4a0c-aaeb-7a80f5bd182e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.501015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.513899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f85b3c-1c92-4a0c-aaeb-7a80f5bd182e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.518098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6c431-0c44-42f9-9b5f-49882f5f11c4 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.554504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e7605-28ee-423a-85a7-dd683d56f4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.555096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7605-28ee-423a-85a7-dd683d56f4df None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.555383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7605-28ee-423a-85a7-dd683d56f4df None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.555665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60e7605-28ee-423a-85a7-dd683d56f4df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.561915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a42be-d0c3-45d6-8595-e9f0af71a7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.562782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a42be-d0c3-45d6-8595-e9f0af71a7ab None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.563235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a42be-d0c3-45d6-8595-e9f0af71a7ab None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.563523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a42be-d0c3-45d6-8595-e9f0af71a7ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.611288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.638040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e7605-28ee-423a-85a7-dd683d56f4df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.639532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60e7605-28ee-423a-85a7-dd683d56f4df None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.645488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a42be-d0c3-45d6-8595-e9f0af71a7ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.647192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a42be-d0c3-45d6-8595-e9f0af71a7ab None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.758291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.777302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 99/199] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:42.778173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a42be-d0c3-45d6-8595-e9f0af71a7ab None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.787691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 101/200] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:42.789535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e7605-28ee-423a-85a7-dd683d56f4df None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.889277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:42.899482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a42be-d0c3-45d6-8595-e9f0af71a7ab None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.918508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60e7605-28ee-423a-85a7-dd683d56f4df None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:42.938701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce998f-e9f3-4c94-a1f1-e7f93bcf44f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.940316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce998f-e9f3-4c94-a1f1-e7f93bcf44f0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.940707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce998f-e9f3-4c94-a1f1-e7f93bcf44f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.941088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce998f-e9f3-4c94-a1f1-e7f93bcf44f0 None admin] PATH_INFO: `/v3/users/7473f83f775a42ce87b115b180c8e94f` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.942617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce998f-e9f3-4c94-a1f1-e7f93bcf44f0 None admin] RBAC: Authorizing `identity:get_user(user_id=7473f83f775a42ce87b115b180c8e94f)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.945067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce998f-e9f3-4c94-a1f1-e7f93bcf44f0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.948705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 100/201] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/users/7473f83f775a42ce87b115b180c8e94f => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:42.949852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce998f-e9f3-4c94-a1f1-e7f93bcf44f0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:42.956827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fed4fb-edd3-4e61-aa68-1e6a0db643ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:42.957954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed4fb-edd3-4e61-aa68-1e6a0db643ef None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:42.958410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed4fb-edd3-4e61-aa68-1e6a0db643ef None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:42.958865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fed4fb-edd3-4e61-aa68-1e6a0db643ef None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:42.960314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fed4fb-edd3-4e61-aa68-1e6a0db643ef None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:42.961970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fed4fb-edd3-4e61-aa68-1e6a0db643ef None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:42.965069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 102/202] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:42.966073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fed4fb-edd3-4e61-aa68-1e6a0db643ef None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.022207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.077969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce998f-e9f3-4c94-a1f1-e7f93bcf44f0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.096172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fed4fb-edd3-4e61-aa68-1e6a0db643ef None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.116853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735aa6a2-304a-4064-bfb9-3427bb619d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.118127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa6a2-304a-4064-bfb9-3427bb619d49 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.118604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa6a2-304a-4064-bfb9-3427bb619d49 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.119914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735aa6a2-304a-4064-bfb9-3427bb619d49 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.121352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735aa6a2-304a-4064-bfb9-3427bb619d49 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.123016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735aa6a2-304a-4064-bfb9-3427bb619d49 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.125355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 101/203] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.126413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735aa6a2-304a-4064-bfb9-3427bb619d49 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.140904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af36c2-3e03-47c9-b328-4fa02db32bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.141499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af36c2-3e03-47c9-b328-4fa02db32bee None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.142052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af36c2-3e03-47c9-b328-4fa02db32bee None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.142339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af36c2-3e03-47c9-b328-4fa02db32bee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.158658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.239453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af36c2-3e03-47c9-b328-4fa02db32bee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.240716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735aa6a2-304a-4064-bfb9-3427bb619d49 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.240968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af36c2-3e03-47c9-b328-4fa02db32bee None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.272446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.288005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.288729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.289021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.289326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.290681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7, user.id=0f1cb372d7a641fc9948b38a438106aa) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.291190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.293277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.300935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 102/204] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/role_assignments?user.id=0f1cb372d7a641fc9948b38a438106aa&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.301547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.378459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 103/205] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:43.384247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af36c2-3e03-47c9-b328-4fa02db32bee None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.439095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b1aae-8a89-402e-a7d0-5d43761f222b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.490171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.490566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.491244 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.491530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.491874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.492752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] RBAC: Adding query filter params (role.id=20a552366bf14792b05decae218a79ee, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7, user.id=0f1cb372d7a641fc9948b38a438106aa) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.493087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.495342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.505041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 103/206] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/role_assignments?user.id=0f1cb372d7a641fc9948b38a438106aa&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=20a552366bf14792b05decae218a79ee => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.580297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af36c2-3e03-47c9-b328-4fa02db32bee None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.618622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb2954-1f02-44e1-a639-8881892e1c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.620047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2954-1f02-44e1-a639-8881892e1c08 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.620356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2954-1f02-44e1-a639-8881892e1c08 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.620638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb2954-1f02-44e1-a639-8881892e1c08 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.621846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb2954-1f02-44e1-a639-8881892e1c08 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.623300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb2954-1f02-44e1-a639-8881892e1c08 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.625041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 104/207] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:43.631074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:43.713337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:43.774803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9328b79-d68e-42d0-ab60-8d620c2fdf63 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:43.817251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2e6d9-2ad5-4c75-90a6-80109a9068c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:43.817937 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2e6d9-2ad5-4c75-90a6-80109a9068c0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:43.818220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2e6d9-2ad5-4c75-90a6-80109a9068c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:43.818495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2e6d9-2ad5-4c75-90a6-80109a9068c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:43.819359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2e6d9-2ad5-4c75-90a6-80109a9068c0 None admin] RBAC: Adding query filter params (role.id=f77ca21ddc4e463e98fbd3e9e7da7512, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd, user.id=7473f83f775a42ce87b115b180c8e94f) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:43.819673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2e6d9-2ad5-4c75-90a6-80109a9068c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:43.821217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2e6d9-2ad5-4c75-90a6-80109a9068c0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:43.829209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 104/208] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:43 2022] GET /identity/v3/role_assignments?user.id=7473f83f775a42ce87b115b180c8e94f&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=f77ca21ddc4e463e98fbd3e9e7da7512 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:44.801736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609d41-e893-4dcb-aa97-4b56f9fa0227 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.802237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609d41-e893-4dcb-aa97-4b56f9fa0227 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.802335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3609d41-e893-4dcb-aa97-4b56f9fa0227 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.803702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 105/209] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:44.809849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380451fd-7f0c-40ef-bd7a-5eec121d64d9 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.810296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380451fd-7f0c-40ef-bd7a-5eec121d64d9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.810464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380451fd-7f0c-40ef-bd7a-5eec121d64d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.829448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-380451fd-7f0c-40ef-bd7a-5eec121d64d9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:44.830645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-380451fd-7f0c-40ef-bd7a-5eec121d64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:44.902452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 105/210] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:44.925899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3609d41-e893-4dcb-aa97-4b56f9fa0227 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:44.944757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7869-473f-468c-b891-861930ab4504 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.945341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7869-473f-468c-b891-861930ab4504 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.945630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f7869-473f-468c-b891-861930ab4504 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.947927 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 106/211] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:44.954059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2eb9-1e04-4aa9-b81a-62828a03376c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:44.955139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2eb9-1e04-4aa9-b81a-62828a03376c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:44.955739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac2eb9-1e04-4aa9-b81a-62828a03376c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:44.975670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3dac2eb9-1e04-4aa9-b81a-62828a03376c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:44.977085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3dac2eb9-1e04-4aa9-b81a-62828a03376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:45.027720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.057432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3609d41-e893-4dcb-aa97-4b56f9fa0227 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.063236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 107/212] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:45.072041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.086795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac2eb9-1e04-4aa9-b81a-62828a03376c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.097436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f72c62-bc77-43da-aa38-363b70e60b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.098576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72c62-bc77-43da-aa38-363b70e60b13 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.099031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72c62-bc77-43da-aa38-363b70e60b13 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.099550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f72c62-bc77-43da-aa38-363b70e60b13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.101825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f72c62-bc77-43da-aa38-363b70e60b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.104240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f72c62-bc77-43da-aa38-363b70e60b13 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.106159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 106/213] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:45.113598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f72c62-bc77-43da-aa38-363b70e60b13 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.191258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac2eb9-1e04-4aa9-b81a-62828a03376c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.225150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f72c62-bc77-43da-aa38-363b70e60b13 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.227191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fa7a4-bafa-4835-9997-e66223313cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.228722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fa7a4-bafa-4835-9997-e66223313cc6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.228722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fa7a4-bafa-4835-9997-e66223313cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.228722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08fa7a4-bafa-4835-9997-e66223313cc6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.239127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08fa7a4-bafa-4835-9997-e66223313cc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c08fa7a4-bafa-4835-9997-e66223313cc6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:45.247010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:45.247894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:45.248703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 108/214] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:45.254329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fa7a4-bafa-4835-9997-e66223313cc6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.265873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6c6432-405d-49f4-9b55-5b8673745beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.266887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c6432-405d-49f4-9b55-5b8673745beb None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.267311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c6432-405d-49f4-9b55-5b8673745beb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.267727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c6432-405d-49f4-9b55-5b8673745beb None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.268816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c6432-405d-49f4-9b55-5b8673745beb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.270468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6c6432-405d-49f4-9b55-5b8673745beb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.292081 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.356635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08fa7a4-bafa-4835-9997-e66223313cc6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.391451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.392289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.392580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.392862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.393621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] RBAC: Adding query filter params (name=member) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:45.393944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.395362 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.404059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 109/215] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:45.409512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.430569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 107/216] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:45 2022] POST /identity/v3/users => generated 297 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:45.520262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1e579-8ead-4955-bc9d-c11e68971451 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.520585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1e579-8ead-4955-bc9d-c11e68971451 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.520820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1e579-8ead-4955-bc9d-c11e68971451 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.522208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 108/217] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:45.527370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.528708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddf4cb-9b6a-4501-9da9-580e3848a346 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.528999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddf4cb-9b6a-4501-9da9-580e3848a346 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.529292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddf4cb-9b6a-4501-9da9-580e3848a346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.534248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac79d75-48cd-44e6-bf00-03cb207d7e55 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.552762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f4ddf4cb-9b6a-4501-9da9-580e3848a346 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:45.553879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f4ddf4cb-9b6a-4501-9da9-580e3848a346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:45.579614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28b218-92c3-4e08-b73f-647577bbe670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.581239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28b218-92c3-4e08-b73f-647577bbe670 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.581719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28b218-92c3-4e08-b73f-647577bbe670 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.582131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28b218-92c3-4e08-b73f-647577bbe670 None admin] PATH_INFO: `/v3/users/bb785627a46341bc87f1347f0caa1e2f` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.584432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.600263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28b218-92c3-4e08-b73f-647577bbe670 None admin] RBAC: Authorizing `identity:get_user(user_id=bb785627a46341bc87f1347f0caa1e2f)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.602056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28b218-92c3-4e08-b73f-647577bbe670 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.605151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 110/218] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/users/bb785627a46341bc87f1347f0caa1e2f => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:45.611029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28b218-92c3-4e08-b73f-647577bbe670 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.631232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 109/219] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:45.655834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddf4cb-9b6a-4501-9da9-580e3848a346 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.721313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28b218-92c3-4e08-b73f-647577bbe670 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.780951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395e699-773a-4ad1-aff3-6fa049524db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.781605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e699-773a-4ad1-aff3-6fa049524db0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.781893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e699-773a-4ad1-aff3-6fa049524db0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.782175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e699-773a-4ad1-aff3-6fa049524db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.783658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddf4cb-9b6a-4501-9da9-580e3848a346 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:45.811717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.828705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1aee8-6794-4002-9e2d-c358b679fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:45.829516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1aee8-6794-4002-9e2d-c358b679fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.829801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1aee8-6794-4002-9e2d-c358b679fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.830358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1aee8-6794-4002-9e2d-c358b679fbf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.842325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1aee8-6794-4002-9e2d-c358b679fbf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-3dd1aee8-6794-4002-9e2d-c358b679fbf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:45.851332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:45.852268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:45.853098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 110/220] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:45.854709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93477d67-1b5e-46e0-90c2-bca397a8ecc0 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.855005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93477d67-1b5e-46e0-90c2-bca397a8ecc0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.855289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93477d67-1b5e-46e0-90c2-bca397a8ecc0 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:45.856771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 111/221] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:45.860041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93477d67-1b5e-46e0-90c2-bca397a8ecc0 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:45.865024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395e699-773a-4ad1-aff3-6fa049524db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:45.866494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395e699-773a-4ad1-aff3-6fa049524db0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:45.921726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:45.994036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 111/222] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:45.995566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe90bcd-01ae-4229-93a1-bf1088c7a938 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:45.995934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe90bcd-01ae-4229-93a1-bf1088c7a938 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:45.996308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe90bcd-01ae-4229-93a1-bf1088c7a938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.007398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93477d67-1b5e-46e0-90c2-bca397a8ecc0 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.018976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8fe90bcd-01ae-4229-93a1-bf1088c7a938 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:46.020294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8fe90bcd-01ae-4229-93a1-bf1088c7a938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:46.047593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.048310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.048590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.048869 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.049815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:46.050126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.051579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.062134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 112/223] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.062917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.092551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.096202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 112/224] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:46.101088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe90bcd-01ae-4229-93a1-bf1088c7a938 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.159073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.175067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dadd7-1837-47f8-aef9-64907086d61f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.211959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b060264-6822-477f-8cdb-e13c593657c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.213515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060264-6822-477f-8cdb-e13c593657c0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.213923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060264-6822-477f-8cdb-e13c593657c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.214209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060264-6822-477f-8cdb-e13c593657c0 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.215434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b060264-6822-477f-8cdb-e13c593657c0 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.216941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b060264-6822-477f-8cdb-e13c593657c0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.218657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 113/225] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.219422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b060264-6822-477f-8cdb-e13c593657c0 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.230993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe90bcd-01ae-4229-93a1-bf1088c7a938 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.279268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b430e-da27-47d3-822e-5ed9c850e459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.280436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b430e-da27-47d3-822e-5ed9c850e459 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.280723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b430e-da27-47d3-822e-5ed9c850e459 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.280999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b430e-da27-47d3-822e-5ed9c850e459 None admin] PATH_INFO: `/v3/users/0f1cb372d7a641fc9948b38a438106aa` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.282618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b430e-da27-47d3-822e-5ed9c850e459 None admin] RBAC: Authorizing `identity:get_user(user_id=0f1cb372d7a641fc9948b38a438106aa)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.284060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b430e-da27-47d3-822e-5ed9c850e459 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.286163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 113/226] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/users/0f1cb372d7a641fc9948b38a438106aa => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.288921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b430e-da27-47d3-822e-5ed9c850e459 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.350418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b060264-6822-477f-8cdb-e13c593657c0 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.355202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.391426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9aa7c-96fc-483b-af6e-6b210834adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.392481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9aa7c-96fc-483b-af6e-6b210834adc3 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.392863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9aa7c-96fc-483b-af6e-6b210834adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.393238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9aa7c-96fc-483b-af6e-6b210834adc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.404664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9aa7c-96fc-483b-af6e-6b210834adc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:46.409792 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b430e-da27-47d3-822e-5ed9c850e459 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-d8a9aa7c-96fc-483b-af6e-6b210834adc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:46.413309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:46.414387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:46.415389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:46.422685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 114/227] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:46.423696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9aa7c-96fc-483b-af6e-6b210834adc3 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.440846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.451150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.451910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.452223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.453915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.453915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] RBAC: Adding query filter params (role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7, user.id=bb785627a46341bc87f1347f0caa1e2f) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:46.454169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.455990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.472521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 114/228] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/role_assignments?user.id=bb785627a46341bc87f1347f0caa1e2f&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.473758 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.543583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9aa7c-96fc-483b-af6e-6b210834adc3 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.585825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef0730-d670-4d27-b6ce-834d010b9756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.585825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0730-d670-4d27-b6ce-834d010b9756 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.585825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0730-d670-4d27-b6ce-834d010b9756 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.586180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef0730-d670-4d27-b6ce-834d010b9756 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.587728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef0730-d670-4d27-b6ce-834d010b9756 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.589258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef0730-d670-4d27-b6ce-834d010b9756 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.594322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 115/229] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.612523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef0730-d670-4d27-b6ce-834d010b9756 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.690784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b4bee-81f8-46b6-bef0-c938a846065d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.719842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.756067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:46.757238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.759076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.759902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.760206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.760981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:46.761290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.762689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.788287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 115/230] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:46.793010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:46.798414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef0730-d670-4d27-b6ce-834d010b9756 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.847679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684ce0c-18be-432a-8f37-acd03f9b45b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.848661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684ce0c-18be-432a-8f37-acd03f9b45b2 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.848957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684ce0c-18be-432a-8f37-acd03f9b45b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.849218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684ce0c-18be-432a-8f37-acd03f9b45b2 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/users/0f1cb372d7a641fc9948b38a438106aa/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.859846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684ce0c-18be-432a-8f37-acd03f9b45b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a1c784d8b9e4949be326e17465c7aa7, user_id=0f1cb372d7a641fc9948b38a438106aa, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.861229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684ce0c-18be-432a-8f37-acd03f9b45b2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.943430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefb47-c0a9-4893-8b40-f824688ef168 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:46.954808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 116/231] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:46 2022] PUT /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/users/0f1cb372d7a641fc9948b38a438106aa/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:46.983036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69aa26-7014-4ed9-8103-6ea9f635df11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:46.984474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69aa26-7014-4ed9-8103-6ea9f635df11 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:46.984963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69aa26-7014-4ed9-8103-6ea9f635df11 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:46.985437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69aa26-7014-4ed9-8103-6ea9f635df11 None admin] PATH_INFO: `/v3/users/0f1cb372d7a641fc9948b38a438106aa` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:46.986978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69aa26-7014-4ed9-8103-6ea9f635df11 None admin] RBAC: Authorizing `identity:get_user(user_id=0f1cb372d7a641fc9948b38a438106aa)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:46.988677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69aa26-7014-4ed9-8103-6ea9f635df11 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:46.990628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 116/232] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/users/0f1cb372d7a641fc9948b38a438106aa => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:46.996794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684ce0c-18be-432a-8f37-acd03f9b45b2 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.110426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684ce0c-18be-432a-8f37-acd03f9b45b2 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.148832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595fed-58e8-4923-a2a8-adabc730ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.150422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595fed-58e8-4923-a2a8-adabc730ba96 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.150909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595fed-58e8-4923-a2a8-adabc730ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.151394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595fed-58e8-4923-a2a8-adabc730ba96 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.152875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea595fed-58e8-4923-a2a8-adabc730ba96 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.154560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea595fed-58e8-4923-a2a8-adabc730ba96 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.156849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 117/233] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:46 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:47.162894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69aa26-7014-4ed9-8103-6ea9f635df11 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:47.225802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:47.276464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69aa26-7014-4ed9-8103-6ea9f635df11 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:47.311834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938bfc69-b61b-4554-8eed-2de940236792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:47.312788 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bfc69-b61b-4554-8eed-2de940236792 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.313084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bfc69-b61b-4554-8eed-2de940236792 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.313366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938bfc69-b61b-4554-8eed-2de940236792 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/users/0f1cb372d7a641fc9948b38a438106aa/roles/20a552366bf14792b05decae218a79ee` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.315625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938bfc69-b61b-4554-8eed-2de940236792 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a1c784d8b9e4949be326e17465c7aa7, user_id=0f1cb372d7a641fc9948b38a438106aa, role_id=20a552366bf14792b05decae218a79ee)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:47.317159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938bfc69-b61b-4554-8eed-2de940236792 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:47.422708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 117/234] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:47 2022] PUT /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/users/0f1cb372d7a641fc9948b38a438106aa/roles/20a552366bf14792b05decae218a79ee => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:40:47.926014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69598ac7-3961-411d-9693-2233d899bf29 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.926520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69598ac7-3961-411d-9693-2233d899bf29 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.926626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69598ac7-3961-411d-9693-2233d899bf29 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.928188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 118/235] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:47.934487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141c88-0003-43d5-b84b-3a8e1e40ec48 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.935010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141c88-0003-43d5-b84b-3a8e1e40ec48 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.935485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141c88-0003-43d5-b84b-3a8e1e40ec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.953923 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-de141c88-0003-43d5-b84b-3a8e1e40ec48 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:47.955830 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-de141c88-0003-43d5-b84b-3a8e1e40ec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:47.973286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1561a2-a867-40dd-9178-3e0fad8fdaa4 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.973580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1561a2-a867-40dd-9178-3e0fad8fdaa4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.973843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1561a2-a867-40dd-9178-3e0fad8fdaa4 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:47.975228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 119/236] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:47.987304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85317-b89b-43be-b763-e68e64e8c3d0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:47.987586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85317-b89b-43be-b763-e68e64e8c3d0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:47.987863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85317-b89b-43be-b763-e68e64e8c3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.014030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-45b85317-b89b-43be-b763-e68e64e8c3d0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:48.015250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-45b85317-b89b-43be-b763-e68e64e8c3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:48.023929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.065527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 118/237] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:48.092710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de141c88-0003-43d5-b84b-3a8e1e40ec48 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.133103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.178541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 120/238] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:48.205717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de141c88-0003-43d5-b84b-3a8e1e40ec48 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.218089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85317-b89b-43be-b763-e68e64e8c3d0 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.268833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc4649-ea78-4fce-ad9d-c5016dcc705d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.270006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc4649-ea78-4fce-ad9d-c5016dcc705d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.270486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc4649-ea78-4fce-ad9d-c5016dcc705d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.270975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc4649-ea78-4fce-ad9d-c5016dcc705d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.272432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc4649-ea78-4fce-ad9d-c5016dcc705d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.274151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc4649-ea78-4fce-ad9d-c5016dcc705d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.276549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 119/239] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:48.281502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc4649-ea78-4fce-ad9d-c5016dcc705d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.334337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85317-b89b-43be-b763-e68e64e8c3d0 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.378867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6d868-5f54-4277-b199-3a984126dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.379675 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6d868-5f54-4277-b199-3a984126dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.379955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6d868-5f54-4277-b199-3a984126dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.380259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6d868-5f54-4277-b199-3a984126dcb6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.400763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.407781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6d868-5f54-4277-b199-3a984126dcb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-f5f6d868-5f54-4277-b199-3a984126dcb6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:48.430178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:48.431227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:48.432074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:48.435710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 121/240] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:48.445752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6d868-5f54-4277-b199-3a984126dcb6 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.492355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc4649-ea78-4fce-ad9d-c5016dcc705d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.501933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.566096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d85fc-b2bf-4681-a7f9-3797f7360953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.566871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d85fc-b2bf-4681-a7f9-3797f7360953 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.567002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d85fc-b2bf-4681-a7f9-3797f7360953 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.567251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d85fc-b2bf-4681-a7f9-3797f7360953 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.567984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d85fc-b2bf-4681-a7f9-3797f7360953 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.569479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d85fc-b2bf-4681-a7f9-3797f7360953 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.584818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6d868-5f54-4277-b199-3a984126dcb6 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.624296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.625301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.625764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.626192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.627138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:48.627593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.629166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.638542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 122/241] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:48.644066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.702705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 120/242] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:48 2022] POST /identity/v3/users => generated 293 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:40:48.713283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:48.758667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030bd44-ddb3-4a0c-8e5c-0543bf936ab6 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:48.796676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7450c20-b393-4580-939c-94556fcf0096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:48.797572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7450c20-b393-4580-939c-94556fcf0096 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.797856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7450c20-b393-4580-939c-94556fcf0096 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.798206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7450c20-b393-4580-939c-94556fcf0096 None admin] PATH_INFO: `/v3/users/c639cb172b23440f82e1fb1933470db5` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.814142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7450c20-b393-4580-939c-94556fcf0096 None admin] RBAC: Authorizing `identity:get_user(user_id=c639cb172b23440f82e1fb1933470db5)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:48.815708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7450c20-b393-4580-939c-94556fcf0096 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:48.817932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 123/243] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/users/c639cb172b23440f82e1fb1933470db5 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:48.824241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d85fc-b2bf-4681-a7f9-3797f7360953 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:48.865965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e06a-4cf2-4bed-8ef0-6bf2e1e63ee5 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.866318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e06a-4cf2-4bed-8ef0-6bf2e1e63ee5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.866534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70e06a-4cf2-4bed-8ef0-6bf2e1e63ee5 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.867940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 124/244] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:48.874905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcadb8c-64c1-4c5c-983c-a9e30f8752cc None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:48.875201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcadb8c-64c1-4c5c-983c-a9e30f8752cc None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:48.875475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcadb8c-64c1-4c5c-983c-a9e30f8752cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:48.894680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0dcadb8c-64c1-4c5c-983c-a9e30f8752cc None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:48.895905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0dcadb8c-64c1-4c5c-983c-a9e30f8752cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:48.983578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 125/245] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:48.991398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d85fc-b2bf-4681-a7f9-3797f7360953 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.006836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcadb8c-64c1-4c5c-983c-a9e30f8752cc None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.033404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484bda2-3ca6-4e86-bebd-b109a9f586bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.034398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484bda2-3ca6-4e86-bebd-b109a9f586bc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.034864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484bda2-3ca6-4e86-bebd-b109a9f586bc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.035319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484bda2-3ca6-4e86-bebd-b109a9f586bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.067693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.069438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.111993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484bda2-3ca6-4e86-bebd-b109a9f586bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.113164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484bda2-3ca6-4e86-bebd-b109a9f586bc None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.131771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcadb8c-64c1-4c5c-983c-a9e30f8752cc None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.175261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4239e94-6ea7-493f-ab41-161cdc990c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.176146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239e94-6ea7-493f-ab41-161cdc990c02 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.176629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239e94-6ea7-493f-ab41-161cdc990c02 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.176982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239e94-6ea7-493f-ab41-161cdc990c02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.192377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4239e94-6ea7-493f-ab41-161cdc990c02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-c4239e94-6ea7-493f-ab41-161cdc990c02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:49.203628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:49.204562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:49.205720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 126/246] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:49.212020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4239e94-6ea7-493f-ab41-161cdc990c02 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.245433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 121/247] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:49.250709 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484bda2-3ca6-4e86-bebd-b109a9f586bc None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.331832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.358013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4239e94-6ea7-493f-ab41-161cdc990c02 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.371930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.394105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484bda2-3ca6-4e86-bebd-b109a9f586bc None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.405902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.407332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.407831 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.408483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.409641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:49.410170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.412730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.427989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 127/248] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:49.433971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.450553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15179db0-22f8-458c-9894-e45703097a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.450553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179db0-22f8-458c-9894-e45703097a8b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.450553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179db0-22f8-458c-9894-e45703097a8b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.450897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179db0-22f8-458c-9894-e45703097a8b None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.451982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15179db0-22f8-458c-9894-e45703097a8b None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.454337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15179db0-22f8-458c-9894-e45703097a8b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.456803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 122/249] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:49.467932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15179db0-22f8-458c-9894-e45703097a8b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.573321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d3a00-4af6-4158-adbb-6e222b6fffd0 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.601267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15179db0-22f8-458c-9894-e45703097a8b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.621781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ebb8f-c450-4803-81b8-d3a47cf8596d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.623082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ebb8f-c450-4803-81b8-d3a47cf8596d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.623639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ebb8f-c450-4803-81b8-d3a47cf8596d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.624182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ebb8f-c450-4803-81b8-d3a47cf8596d None admin] PATH_INFO: `/v3/users/bb785627a46341bc87f1347f0caa1e2f` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.627285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ebb8f-c450-4803-81b8-d3a47cf8596d None admin] RBAC: Authorizing `identity:get_user(user_id=bb785627a46341bc87f1347f0caa1e2f)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.630507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ebb8f-c450-4803-81b8-d3a47cf8596d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.636540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 128/250] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/users/bb785627a46341bc87f1347f0caa1e2f => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:49.648268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ebb8f-c450-4803-81b8-d3a47cf8596d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.654178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.671117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdc91d-0aa8-496a-9679-0013f8cc7dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.672408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc91d-0aa8-496a-9679-0013f8cc7dbf None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.672864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc91d-0aa8-496a-9679-0013f8cc7dbf None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.676046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc91d-0aa8-496a-9679-0013f8cc7dbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.676046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdc91d-0aa8-496a-9679-0013f8cc7dbf None admin] RBAC: Adding query filter params (role.id=f77ca21ddc4e463e98fbd3e9e7da7512, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7, user.id=c639cb172b23440f82e1fb1933470db5) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:49.676046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdc91d-0aa8-496a-9679-0013f8cc7dbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.677047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdc91d-0aa8-496a-9679-0013f8cc7dbf None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.692736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 123/251] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/role_assignments?user.id=c639cb172b23440f82e1fb1933470db5&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=f77ca21ddc4e463e98fbd3e9e7da7512 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:49.705727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:49.721390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f339b1-5518-4fc8-958b-d1ed87d17032 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.722008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f339b1-5518-4fc8-958b-d1ed87d17032 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.722803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f339b1-5518-4fc8-958b-d1ed87d17032 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.724351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 124/252] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:49.730429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bef031-6f1b-4ad5-a668-0ad034d6edaa None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.731003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bef031-6f1b-4ad5-a668-0ad034d6edaa None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.731491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bef031-6f1b-4ad5-a668-0ad034d6edaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.768381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-40bef031-6f1b-4ad5-a668-0ad034d6edaa None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:49.769968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-40bef031-6f1b-4ad5-a668-0ad034d6edaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:49.789153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ebb8f-c450-4803-81b8-d3a47cf8596d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:49.834107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461625d2-aca0-4f53-bdd9-ffe8ad9b6880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:49.835162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461625d2-aca0-4f53-bdd9-ffe8ad9b6880 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:49.836265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461625d2-aca0-4f53-bdd9-ffe8ad9b6880 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:49.836665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461625d2-aca0-4f53-bdd9-ffe8ad9b6880 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:49.838941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461625d2-aca0-4f53-bdd9-ffe8ad9b6880 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:49.841559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461625d2-aca0-4f53-bdd9-ffe8ad9b6880 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:49.844267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 129/253] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:49.854561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461625d2-aca0-4f53-bdd9-ffe8ad9b6880 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:49.879996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 125/254] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:49.915598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bef031-6f1b-4ad5-a668-0ad034d6edaa None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.055503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461625d2-aca0-4f53-bdd9-ffe8ad9b6880 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.068525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.098922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e432c34-5ab9-4a76-875e-d39c7d80619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.100160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432c34-5ab9-4a76-875e-d39c7d80619b None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.100991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432c34-5ab9-4a76-875e-d39c7d80619b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.101423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e432c34-5ab9-4a76-875e-d39c7d80619b None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/users/bb785627a46341bc87f1347f0caa1e2f/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.105126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e432c34-5ab9-4a76-875e-d39c7d80619b None admin] RBAC: Authorizing `identity:create_grant(project_id=2a1c784d8b9e4949be326e17465c7aa7, user_id=bb785627a46341bc87f1347f0caa1e2f, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.107645 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e432c34-5ab9-4a76-875e-d39c7d80619b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.129141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bef031-6f1b-4ad5-a668-0ad034d6edaa None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.154130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.165980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a1d23-47e8-4f89-b0c9-d862bd4d9c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.166785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a1d23-47e8-4f89-b0c9-d862bd4d9c6e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.167066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a1d23-47e8-4f89-b0c9-d862bd4d9c6e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.167334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a1d23-47e8-4f89-b0c9-d862bd4d9c6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.177834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a1d23-47e8-4f89-b0c9-d862bd4d9c6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:50.178736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 130/255] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:49 2022] PUT /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/users/bb785627a46341bc87f1347f0caa1e2f/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:40:50.184885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bcce-c844-4463-bfc0-5346d441bf1c None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.185380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bcce-c844-4463-bfc0-5346d441bf1c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.185805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4bcce-c844-4463-bfc0-5346d441bf1c None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-5b2a1d23-47e8-4f89-b0c9-d862bd4d9c6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:50.185984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:50.186988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:50.187953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 126/256] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:40:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:50.190250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 131/257] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:50 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:50.196187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a1d23-47e8-4f89-b0c9-d862bd4d9c6e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.198446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97b61-10cf-4658-87b7-5910e79b222d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.198917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97b61-10cf-4658-87b7-5910e79b222d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.199350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b97b61-10cf-4658-87b7-5910e79b222d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.237217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d0b97b61-10cf-4658-87b7-5910e79b222d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:50.238771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d0b97b61-10cf-4658-87b7-5910e79b222d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:50.302979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a1d23-47e8-4f89-b0c9-d862bd4d9c6e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.340748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.341431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.341717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.342048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.342813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:50.343286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.349250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.355046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 127/258] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:50.364224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.365869 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 132/259] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:50.407431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97b61-10cf-4658-87b7-5910e79b222d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.440523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.472757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e113a-4454-40aa-ba64-009ca91db993 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.510320 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151b0c2-7cb3-4d2b-b0b8-b8e87e856a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.511353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151b0c2-7cb3-4d2b-b0b8-b8e87e856a70 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.511606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151b0c2-7cb3-4d2b-b0b8-b8e87e856a70 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.511894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2151b0c2-7cb3-4d2b-b0b8-b8e87e856a70 None admin] PATH_INFO: `/v3/users/0f1cb372d7a641fc9948b38a438106aa` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.517265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151b0c2-7cb3-4d2b-b0b8-b8e87e856a70 None admin] RBAC: Authorizing `identity:get_user(user_id=0f1cb372d7a641fc9948b38a438106aa)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.518657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2151b0c2-7cb3-4d2b-b0b8-b8e87e856a70 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.520653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 128/260] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/users/0f1cb372d7a641fc9948b38a438106aa => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:50.532890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151b0c2-7cb3-4d2b-b0b8-b8e87e856a70 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.575803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.581802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b97b61-10cf-4658-87b7-5910e79b222d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.623245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d92a1-8e62-44a6-b959-ca4b9cb571bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.624334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d92a1-8e62-44a6-b959-ca4b9cb571bd None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.624783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d92a1-8e62-44a6-b959-ca4b9cb571bd None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.625204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d92a1-8e62-44a6-b959-ca4b9cb571bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.640305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d92a1-8e62-44a6-b959-ca4b9cb571bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:50.647447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2151b0c2-7cb3-4d2b-b0b8-b8e87e856a70 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-d21d92a1-8e62-44a6-b959-ca4b9cb571bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:50.657032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:50.658200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:50.659156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 133/261] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:50.668827 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:50.669045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d92a1-8e62-44a6-b959-ca4b9cb571bd None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:50.700358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb0967-7d46-4c1d-a185-b9b579e8981d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.701257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb0967-7d46-4c1d-a185-b9b579e8981d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.702489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb0967-7d46-4c1d-a185-b9b579e8981d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.703161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb0967-7d46-4c1d-a185-b9b579e8981d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.844834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb0967-7d46-4c1d-a185-b9b579e8981d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.847093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb0967-7d46-4c1d-a185-b9b579e8981d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.891401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d92a1-8e62-44a6-b959-ca4b9cb571bd None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:50.962791 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:50.963475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:50.963759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:50.964036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:50.964827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:50.965136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:50.966548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:50.995694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 134/262] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:51.001713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.041086 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:51.099158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 129/263] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:51.105454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb0967-7d46-4c1d-a185-b9b579e8981d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.135555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:51.180914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca8e12-2e6b-4ca4-9cde-512ec17db516 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.236379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0938f1d-c1f1-4c91-a889-f2a07759123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.237273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0938f1d-c1f1-4c91-a889-f2a07759123b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.237561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0938f1d-c1f1-4c91-a889-f2a07759123b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.237837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0938f1d-c1f1-4c91-a889-f2a07759123b None admin] PATH_INFO: `/v3/users/0f1cb372d7a641fc9948b38a438106aa` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.245110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0938f1d-c1f1-4c91-a889-f2a07759123b None admin] RBAC: Authorizing `identity:get_user(user_id=0f1cb372d7a641fc9948b38a438106aa)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.246640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0938f1d-c1f1-4c91-a889-f2a07759123b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.248821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 135/264] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/users/0f1cb372d7a641fc9948b38a438106aa => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:51.254305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0938f1d-c1f1-4c91-a889-f2a07759123b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.347979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb0967-7d46-4c1d-a185-b9b579e8981d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.365712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:51.379135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0938f1d-c1f1-4c91-a889-f2a07759123b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.419328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443ad49-4488-4579-9e7b-46bc10adf6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.419988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443ad49-4488-4579-9e7b-46bc10adf6bf None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.420304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443ad49-4488-4579-9e7b-46bc10adf6bf None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.420583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0443ad49-4488-4579-9e7b-46bc10adf6bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.424809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7665339-c943-4186-b468-d6cb52571521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.425670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7665339-c943-4186-b468-d6cb52571521 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.425952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7665339-c943-4186-b468-d6cb52571521 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.426231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7665339-c943-4186-b468-d6cb52571521 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.432072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7665339-c943-4186-b468-d6cb52571521 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.433544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7665339-c943-4186-b468-d6cb52571521 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.435738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 130/265] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:51.441068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7665339-c943-4186-b468-d6cb52571521 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.497039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443ad49-4488-4579-9e7b-46bc10adf6bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.498519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0443ad49-4488-4579-9e7b-46bc10adf6bf None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.590464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:51.604299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 136/266] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:40:51.610082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:51.610227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443ad49-4488-4579-9e7b-46bc10adf6bf None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.651001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7665339-c943-4186-b468-d6cb52571521 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.737870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.738465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.738751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.739101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.739947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7, user.id=0f1cb372d7a641fc9948b38a438106aa) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:51.740287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.741744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.762446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 131/267] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/role_assignments?user.id=0f1cb372d7a641fc9948b38a438106aa&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:51.768897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0443ad49-4488-4579-9e7b-46bc10adf6bf None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:51.834287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197d3b3-9411-47ae-a98a-765910a4f82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:51.835524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197d3b3-9411-47ae-a98a-765910a4f82a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:51.836036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197d3b3-9411-47ae-a98a-765910a4f82a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:51.836536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197d3b3-9411-47ae-a98a-765910a4f82a None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:51.840895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197d3b3-9411-47ae-a98a-765910a4f82a None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:51.842627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197d3b3-9411-47ae-a98a-765910a4f82a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:51.844637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 137/268] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:51.851122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:51.997946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c7918-9b4f-4f96-810a-ef68968b6a84 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:52.045451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1c065-873b-4199-8a28-1191b8bda739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:52.046118 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1c065-873b-4199-8a28-1191b8bda739 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.046400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1c065-873b-4199-8a28-1191b8bda739 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.046673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1c065-873b-4199-8a28-1191b8bda739 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.047512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1c065-873b-4199-8a28-1191b8bda739 None admin] RBAC: Adding query filter params (role.id=20a552366bf14792b05decae218a79ee, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7, user.id=0f1cb372d7a641fc9948b38a438106aa) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:52.047824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1c065-873b-4199-8a28-1191b8bda739 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:52.049295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1c065-873b-4199-8a28-1191b8bda739 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:52.061095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 132/269] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:51 2022] GET /identity/v3/role_assignments?user.id=0f1cb372d7a641fc9948b38a438106aa&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=20a552366bf14792b05decae218a79ee => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:52.236712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007e6ad-a07d-474c-b61a-4e5df3e3276b None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.237712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007e6ad-a07d-474c-b61a-4e5df3e3276b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.238203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d007e6ad-a07d-474c-b61a-4e5df3e3276b None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.240011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 138/270] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:52.247627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092d289-8577-484e-87a7-5ddba2414eb6 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.248687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092d289-8577-484e-87a7-5ddba2414eb6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.249189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092d289-8577-484e-87a7-5ddba2414eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.267755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f092d289-8577-484e-87a7-5ddba2414eb6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:52.269365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f092d289-8577-484e-87a7-5ddba2414eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:52.271757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:52.346092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 133/271] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:52.371614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007e6ad-a07d-474c-b61a-4e5df3e3276b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.437245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:52.463658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35301f-7428-4716-bc16-e0817c71ba4f None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.463942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35301f-7428-4716-bc16-e0817c71ba4f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.464247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d35301f-7428-4716-bc16-e0817c71ba4f None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.465844 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 134/272] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:52.471671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5e586-9407-4cfe-a8ab-9eccfd29a7bc None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.471954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5e586-9407-4cfe-a8ab-9eccfd29a7bc None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.472257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5e586-9407-4cfe-a8ab-9eccfd29a7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.502469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-88f5e586-9407-4cfe-a8ab-9eccfd29a7bc None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:52.504001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-88f5e586-9407-4cfe-a8ab-9eccfd29a7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:52.505362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d007e6ad-a07d-474c-b61a-4e5df3e3276b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:52.554162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de2d8b-712c-45be-8c75-bfc74c9a1b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:52.555407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de2d8b-712c-45be-8c75-bfc74c9a1b10 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.556519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de2d8b-712c-45be-8c75-bfc74c9a1b10 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.557017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de2d8b-712c-45be-8c75-bfc74c9a1b10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.569019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de2d8b-712c-45be-8c75-bfc74c9a1b10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-65de2d8b-712c-45be-8c75-bfc74c9a1b10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:52.577792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:52.578690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:52.579463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:52.581877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 139/273] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:52.588587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de2d8b-712c-45be-8c75-bfc74c9a1b10 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.600258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 135/274] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:52.627086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5e586-9407-4cfe-a8ab-9eccfd29a7bc None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.719018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:52.730268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de2d8b-712c-45be-8c75-bfc74c9a1b10 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:52.750414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:52.765707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5e586-9407-4cfe-a8ab-9eccfd29a7bc None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:52.772168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b368917-b7c8-48c5-b28f-dd64e58d91a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:52.772845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368917-b7c8-48c5-b28f-dd64e58d91a6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.773125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368917-b7c8-48c5-b28f-dd64e58d91a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.775503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368917-b7c8-48c5-b28f-dd64e58d91a6 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.775503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b368917-b7c8-48c5-b28f-dd64e58d91a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:52.775503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b368917-b7c8-48c5-b28f-dd64e58d91a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:52.775877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b368917-b7c8-48c5-b28f-dd64e58d91a6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:52.789274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 140/275] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:52.789274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79737d87-73ad-4695-8f42-9713ea6633cf None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.789274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79737d87-73ad-4695-8f42-9713ea6633cf None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.789274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79737d87-73ad-4695-8f42-9713ea6633cf None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.789274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 141/276] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:52.793461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79737d87-73ad-4695-8f42-9713ea6633cf None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.806571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89faa005-c1c2-4cfc-af83-2a37c095000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:52.807626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faa005-c1c2-4cfc-af83-2a37c095000b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.807981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faa005-c1c2-4cfc-af83-2a37c095000b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.808302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89faa005-c1c2-4cfc-af83-2a37c095000b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.813883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89faa005-c1c2-4cfc-af83-2a37c095000b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:52.813883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89faa005-c1c2-4cfc-af83-2a37c095000b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:52.813883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 136/277] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:52.818219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053080a-65c4-48be-a255-c4c605661a76 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.818518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053080a-65c4-48be-a255-c4c605661a76 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.818801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053080a-65c4-48be-a255-c4c605661a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.835732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f053080a-65c4-48be-a255-c4c605661a76 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:52.837001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f053080a-65c4-48be-a255-c4c605661a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:52.904978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79737d87-73ad-4695-8f42-9713ea6633cf None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:52.910490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 137/278] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:52.917083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053080a-65c4-48be-a255-c4c605661a76 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.944163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c82e6-f7ee-49ab-bd6c-ae61af3c6b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:52.945054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c82e6-f7ee-49ab-bd6c-ae61af3c6b89 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:52.945340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c82e6-f7ee-49ab-bd6c-ae61af3c6b89 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:52.945620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c82e6-f7ee-49ab-bd6c-ae61af3c6b89 None admin] PATH_INFO: `/v3/users/a0529e390b764b7bb6c1362f20af5978` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:52.960041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c82e6-f7ee-49ab-bd6c-ae61af3c6b89 None admin] RBAC: Authorizing `identity:get_user(user_id=a0529e390b764b7bb6c1362f20af5978)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:52.961708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c82e6-f7ee-49ab-bd6c-ae61af3c6b89 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:52.964141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 142/279] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/users/a0529e390b764b7bb6c1362f20af5978 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:52.968318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c82e6-f7ee-49ab-bd6c-ae61af3c6b89 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:52.995976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.006920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.033212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053080a-65c4-48be-a255-c4c605661a76 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.068050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a041b2c-f2eb-4688-9117-90c4ba3ccb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.068756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a041b2c-f2eb-4688-9117-90c4ba3ccb3b None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.069044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a041b2c-f2eb-4688-9117-90c4ba3ccb3b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.069324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a041b2c-f2eb-4688-9117-90c4ba3ccb3b None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.070030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a041b2c-f2eb-4688-9117-90c4ba3ccb3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.071533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a041b2c-f2eb-4688-9117-90c4ba3ccb3b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.076472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c82e6-f7ee-49ab-bd6c-ae61af3c6b89 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.111162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf79943-c379-4f60-ac60-26fd48c58c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.111953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf79943-c379-4f60-ac60-26fd48c58c27 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.112271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf79943-c379-4f60-ac60-26fd48c58c27 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.112551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf79943-c379-4f60-ac60-26fd48c58c27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.122945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf79943-c379-4f60-ac60-26fd48c58c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-3cf79943-c379-4f60-ac60-26fd48c58c27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:53.130733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:53.131794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:53.132658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 143/280] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:53.133302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf79943-c379-4f60-ac60-26fd48c58c27 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.171455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 138/281] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:40:52 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:53.181300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a041b2c-f2eb-4688-9117-90c4ba3ccb3b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.256922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.267807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf79943-c379-4f60-ac60-26fd48c58c27 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.306706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c62f26-12e2-49ae-8255-bddc22fdb6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.307326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c62f26-12e2-49ae-8255-bddc22fdb6ef None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.307612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c62f26-12e2-49ae-8255-bddc22fdb6ef None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.307887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c62f26-12e2-49ae-8255-bddc22fdb6ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.308799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c62f26-12e2-49ae-8255-bddc22fdb6ef None admin] RBAC: Adding query filter params (role.id=45ad0499b0b54c309464a2ebb8a3e5d0, user.id=a0529e390b764b7bb6c1362f20af5978, scope.system=all) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:53.309121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c62f26-12e2-49ae-8255-bddc22fdb6ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.310643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c62f26-12e2-49ae-8255-bddc22fdb6ef None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.316704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 144/282] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/role_assignments?user.id=a0529e390b764b7bb6c1362f20af5978&scope.system=all&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:53.322084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a041b2c-f2eb-4688-9117-90c4ba3ccb3b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.352384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.366165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe789-a717-42f5-a0a3-11e6aa2f30dc None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.375381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe789-a717-42f5-a0a3-11e6aa2f30dc None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.375381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe789-a717-42f5-a0a3-11e6aa2f30dc None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.375567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.375567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.375567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.375779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 145/283] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:40:53.375851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.375851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:53.375851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.375851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.376077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18511866-0d7c-45bc-825b-2efb9db3be3a None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.376077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18511866-0d7c-45bc-825b-2efb9db3be3a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.376077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18511866-0d7c-45bc-825b-2efb9db3be3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.381351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 139/284] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:53.388043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.397355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-18511866-0d7c-45bc-825b-2efb9db3be3a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:53.398653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-18511866-0d7c-45bc-825b-2efb9db3be3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:53.512825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 146/285] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:40:53.537262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1d873-0f3a-44e7-a373-b9669c37fa03 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.545045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18511866-0d7c-45bc-825b-2efb9db3be3a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.581787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234de55-1c97-4caa-8870-08bf5247c8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.583069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234de55-1c97-4caa-8870-08bf5247c8dd None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.583615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234de55-1c97-4caa-8870-08bf5247c8dd None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.583906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234de55-1c97-4caa-8870-08bf5247c8dd None admin] PATH_INFO: `/v3/users/c639cb172b23440f82e1fb1933470db5` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.585174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234de55-1c97-4caa-8870-08bf5247c8dd None admin] RBAC: Authorizing `identity:get_user(user_id=c639cb172b23440f82e1fb1933470db5)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.586814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234de55-1c97-4caa-8870-08bf5247c8dd None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.590230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 140/286] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/users/c639cb172b23440f82e1fb1933470db5 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:53.596166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234de55-1c97-4caa-8870-08bf5247c8dd None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.680295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.691204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18511866-0d7c-45bc-825b-2efb9db3be3a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.700799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:53.747051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9c587-0b15-4518-920d-28fdeb6a2376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.747920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9c587-0b15-4518-920d-28fdeb6a2376 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.748240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9c587-0b15-4518-920d-28fdeb6a2376 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.748522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a9c587-0b15-4518-920d-28fdeb6a2376 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.763000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234de55-1c97-4caa-8870-08bf5247c8dd None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.779464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a9c587-0b15-4518-920d-28fdeb6a2376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-c0a9c587-0b15-4518-920d-28fdeb6a2376 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:53.789362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:53.790872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:53.791658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:53.799158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 147/287] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:53.810498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9c587-0b15-4518-920d-28fdeb6a2376 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.838429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111be34-c09b-4d75-b491-b1b8d744184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.839341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111be34-c09b-4d75-b491-b1b8d744184e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.839731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111be34-c09b-4d75-b491-b1b8d744184e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.840132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f111be34-c09b-4d75-b491-b1b8d744184e None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.843227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111be34-c09b-4d75-b491-b1b8d744184e None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.843227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f111be34-c09b-4d75-b491-b1b8d744184e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:53.850159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 141/288] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:53.851425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111be34-c09b-4d75-b491-b1b8d744184e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:53.930276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a9c587-0b15-4518-920d-28fdeb6a2376 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:53.978771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:53.979854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:53.980378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:53.980802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:53.981712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:53.982196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:53.983741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.007138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 148/289] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:54.008320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.039687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.041936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f111be34-c09b-4d75-b491-b1b8d744184e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.064444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.081232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328b390-a936-4914-9a60-be0de50e2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.082400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328b390-a936-4914-9a60-be0de50e2428 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.082812 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328b390-a936-4914-9a60-be0de50e2428 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.083109 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328b390-a936-4914-9a60-be0de50e2428 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/users/c639cb172b23440f82e1fb1933470db5/roles/f77ca21ddc4e463e98fbd3e9e7da7512` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.085401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328b390-a936-4914-9a60-be0de50e2428 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a1c784d8b9e4949be326e17465c7aa7, user_id=c639cb172b23440f82e1fb1933470db5, role_id=f77ca21ddc4e463e98fbd3e9e7da7512)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.087043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328b390-a936-4914-9a60-be0de50e2428 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.164833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 142/290] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:40:53 2022] PUT /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/users/c639cb172b23440f82e1fb1933470db5/roles/f77ca21ddc4e463e98fbd3e9e7da7512 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:40:54.179192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8028c0-6371-4b6b-aad0-dd77083a684d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.218500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84609d-2a81-4b3d-900f-0466ee4cfdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.219397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84609d-2a81-4b3d-900f-0466ee4cfdaa None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.219685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84609d-2a81-4b3d-900f-0466ee4cfdaa None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.219966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84609d-2a81-4b3d-900f-0466ee4cfdaa None admin] PATH_INFO: `/v3/users/bb785627a46341bc87f1347f0caa1e2f` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.221286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84609d-2a81-4b3d-900f-0466ee4cfdaa None admin] RBAC: Authorizing `identity:get_user(user_id=bb785627a46341bc87f1347f0caa1e2f)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.222837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84609d-2a81-4b3d-900f-0466ee4cfdaa None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.224956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 149/291] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/users/bb785627a46341bc87f1347f0caa1e2f => generated 297 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:54.230734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328b390-a936-4914-9a60-be0de50e2428 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.340510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328b390-a936-4914-9a60-be0de50e2428 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.376915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a4feb-3b5f-45ca-b6e6-ce3bdea00649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.377587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a4feb-3b5f-45ca-b6e6-ce3bdea00649 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.377876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a4feb-3b5f-45ca-b6e6-ce3bdea00649 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.378160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a4feb-3b5f-45ca-b6e6-ce3bdea00649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.430152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.452650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a4feb-3b5f-45ca-b6e6-ce3bdea00649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.454139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a4feb-3b5f-45ca-b6e6-ce3bdea00649 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.558051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 143/292] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:54.563566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84609d-2a81-4b3d-900f-0466ee4cfdaa None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.648171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.668008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84609d-2a81-4b3d-900f-0466ee4cfdaa None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.708764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e5390-8bd9-4b66-be61-a59d1081259b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.709558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5390-8bd9-4b66-be61-a59d1081259b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.709844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5390-8bd9-4b66-be61-a59d1081259b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.710125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e5390-8bd9-4b66-be61-a59d1081259b None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.711350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e5390-8bd9-4b66-be61-a59d1081259b None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.713368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e5390-8bd9-4b66-be61-a59d1081259b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.715112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 150/293] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:54.721005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a4feb-3b5f-45ca-b6e6-ce3bdea00649 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:54.834112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:54.856708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a4feb-3b5f-45ca-b6e6-ce3bdea00649 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:54.906673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bdd3f-afdb-466d-92cd-53017972c99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:54.907393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bdd3f-afdb-466d-92cd-53017972c99f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:54.907684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bdd3f-afdb-466d-92cd-53017972c99f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:54.907963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bdd3f-afdb-466d-92cd-53017972c99f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:54.909033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bdd3f-afdb-466d-92cd-53017972c99f None admin] RBAC: Adding query filter params (role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7, user.id=bb785627a46341bc87f1347f0caa1e2f) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:54.909349 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bdd3f-afdb-466d-92cd-53017972c99f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:54.910787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bdd3f-afdb-466d-92cd-53017972c99f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:54.920862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 144/294] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:54 2022] GET /identity/v3/role_assignments?user.id=bb785627a46341bc87f1347f0caa1e2f&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:55.662178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749882c-16b6-483b-b804-b256672c7c8e None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.662690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749882c-16b6-483b-b804-b256672c7c8e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.662790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8749882c-16b6-483b-b804-b256672c7c8e None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.664107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 151/295] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:55.669673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32aee1-7550-4715-b3e6-dd88861289e4 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.670092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32aee1-7550-4715-b3e6-dd88861289e4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.670267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32aee1-7550-4715-b3e6-dd88861289e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.687028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3b32aee1-7550-4715-b3e6-dd88861289e4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:55.688595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3b32aee1-7550-4715-b3e6-dd88861289e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:55.761992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 145/296] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:55.791474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728caf5-da47-442b-a6aa-1c1ad61d7d31 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.792140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728caf5-da47-442b-a6aa-1c1ad61d7d31 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.792715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728caf5-da47-442b-a6aa-1c1ad61d7d31 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.793588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32aee1-7550-4715-b3e6-dd88861289e4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:55.794795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 152/297] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:55.800971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e5fe8-8b2b-4ccf-97a7-b3f53c5fb278 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.801538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e5fe8-8b2b-4ccf-97a7-b3f53c5fb278 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.801996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e5fe8-8b2b-4ccf-97a7-b3f53c5fb278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.820059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b10e5fe8-8b2b-4ccf-97a7-b3f53c5fb278 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:55.822265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b10e5fe8-8b2b-4ccf-97a7-b3f53c5fb278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:55.872501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:55.898434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 153/298] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:55.898597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32aee1-7550-4715-b3e6-dd88861289e4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:55.900324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3be0e-f957-412e-9920-287b4231df50 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.900825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3be0e-f957-412e-9920-287b4231df50 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.901301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3be0e-f957-412e-9920-287b4231df50 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.902794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 154/299] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:55.913985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a8c0d-a858-4769-98ce-bd2991262dfb None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.914561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a8c0d-a858-4769-98ce-bd2991262dfb None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.915070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a8c0d-a858-4769-98ce-bd2991262dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.932750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-3f2a8c0d-a858-4769-98ce-bd2991262dfb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:55.934019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3f2a8c0d-a858-4769-98ce-bd2991262dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:55.935376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cb168-0541-49cb-86d6-60cbdac06a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:55.936213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cb168-0541-49cb-86d6-60cbdac06a6c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:55.936502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cb168-0541-49cb-86d6-60cbdac06a6c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:55.936777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769cb168-0541-49cb-86d6-60cbdac06a6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:55.938126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cb168-0541-49cb-86d6-60cbdac06a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:55.939574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769cb168-0541-49cb-86d6-60cbdac06a6c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:55.941411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 146/300] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:55.947052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cb168-0541-49cb-86d6-60cbdac06a6c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.005452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 155/301] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:56.017017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a8c0d-a858-4769-98ce-bd2991262dfb None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.029911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.053608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769cb168-0541-49cb-86d6-60cbdac06a6c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.090792 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a114e-0369-433e-9b9a-15d97c65d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.091593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a114e-0369-433e-9b9a-15d97c65d514 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.091879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a114e-0369-433e-9b9a-15d97c65d514 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.092303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a114e-0369-433e-9b9a-15d97c65d514 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.103168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a114e-0369-433e-9b9a-15d97c65d514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-087a114e-0369-433e-9b9a-15d97c65d514 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:56.111053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:56.112141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:56.112982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 147/302] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:56.113644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a114e-0369-433e-9b9a-15d97c65d514 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.119552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.130499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a8c0d-a858-4769-98ce-bd2991262dfb None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.168810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914462-ccd3-4c66-9456-27dc9eebee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.169629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914462-ccd3-4c66-9456-27dc9eebee73 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.169916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914462-ccd3-4c66-9456-27dc9eebee73 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.170195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a914462-ccd3-4c66-9456-27dc9eebee73 None admin] PATH_INFO: `/v3/groups` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.170889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a914462-ccd3-4c66-9456-27dc9eebee73 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.172499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a914462-ccd3-4c66-9456-27dc9eebee73 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.218441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a114e-0369-433e-9b9a-15d97c65d514 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.250290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 156/303] 10.222.0.40 () {60 vars in 1134 bytes} [Mon Jul 25 05:40:56 2022] POST /identity/v3/groups => generated 229 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:56.251426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914462-ccd3-4c66-9456-27dc9eebee73 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.258334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625c937-4be1-4df4-b77a-c2d32d72169b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.260525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625c937-4be1-4df4-b77a-c2d32d72169b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.261038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625c937-4be1-4df4-b77a-c2d32d72169b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.261586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625c937-4be1-4df4-b77a-c2d32d72169b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.280633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625c937-4be1-4df4-b77a-c2d32d72169b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-8625c937-4be1-4df4-b77a-c2d32d72169b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:56.292629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:56.293843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:56.294872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:56.298342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 148/304] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:56.304453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625c937-4be1-4df4-b77a-c2d32d72169b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.342652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.383361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a914462-ccd3-4c66-9456-27dc9eebee73 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.407779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.431986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.432959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.433254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.433538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.434312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:56.434628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.436095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.449509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 157/305] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:56.455817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.460851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625c937-4be1-4df4-b77a-c2d32d72169b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.513409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.514933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.515535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.516014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.516994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:56.517503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.519331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.529592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 149/306] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:56.534871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.573481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ec908-40ed-4457-9a84-6fb736e923c5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.605022 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.611070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5a39-8513-4291-bc6b-d12160cdcc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.612390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5a39-8513-4291-bc6b-d12160cdcc1e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.615479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5a39-8513-4291-bc6b-d12160cdcc1e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.615479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6a5a39-8513-4291-bc6b-d12160cdcc1e None admin] PATH_INFO: `/v3/users/a0529e390b764b7bb6c1362f20af5978` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.615479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5a39-8513-4291-bc6b-d12160cdcc1e None admin] RBAC: Authorizing `identity:get_user(user_id=a0529e390b764b7bb6c1362f20af5978)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.617276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6a5a39-8513-4291-bc6b-d12160cdcc1e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.619968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 158/307] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/users/a0529e390b764b7bb6c1362f20af5978 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:56.625887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5a39-8513-4291-bc6b-d12160cdcc1e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.672453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224aaeaf-35f3-4a75-973f-952dfa0eb5f5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.704872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.718711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f62d3a-a665-4f71-9594-f3b05f16785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.719603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62d3a-a665-4f71-9594-f3b05f16785d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.719891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62d3a-a665-4f71-9594-f3b05f16785d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.720205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f62d3a-a665-4f71-9594-f3b05f16785d None admin] PATH_INFO: `/v3/users/9b912d49c8a84405aa2cc61938024eb6` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.736900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f62d3a-a665-4f71-9594-f3b05f16785d None admin] RBAC: Authorizing `identity:get_user(user_id=9b912d49c8a84405aa2cc61938024eb6)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.738905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f62d3a-a665-4f71-9594-f3b05f16785d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.740966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 150/308] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/users/9b912d49c8a84405aa2cc61938024eb6 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:56.742234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cdac-7feb-478d-a232-53e14340e476 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.742522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cdac-7feb-478d-a232-53e14340e476 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.742797 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4cdac-7feb-478d-a232-53e14340e476 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.744798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 151/309] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:56.750401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4cdac-7feb-478d-a232-53e14340e476 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:56.754805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6a5a39-8513-4291-bc6b-d12160cdcc1e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.798237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468973c-41c1-4d68-8bac-6a6de21ce78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.799506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468973c-41c1-4d68-8bac-6a6de21ce78c None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.799799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468973c-41c1-4d68-8bac-6a6de21ce78c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.800083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468973c-41c1-4d68-8bac-6a6de21ce78c None admin] PATH_INFO: `/v3/system/users/a0529e390b764b7bb6c1362f20af5978/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.801656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468973c-41c1-4d68-8bac-6a6de21ce78c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a0529e390b764b7bb6c1362f20af5978, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.803059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468973c-41c1-4d68-8bac-6a6de21ce78c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.858574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4cdac-7feb-478d-a232-53e14340e476 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:56.861843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 159/310] 10.222.0.40 () {58 vars in 1334 bytes} [Mon Jul 25 05:40:56 2022] PUT /identity/v3/system/users/a0529e390b764b7bb6c1362f20af5978/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:40:56.863345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88b01-342b-4b2c-8788-af7f465f4ee1 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.863740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88b01-342b-4b2c-8788-af7f465f4ee1 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.864088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc88b01-342b-4b2c-8788-af7f465f4ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.886894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4bc88b01-342b-4b2c-8788-af7f465f4ee1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:56.888172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4bc88b01-342b-4b2c-8788-af7f465f4ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:56.919145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:56.920328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:56.920618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:56.921418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:56.922617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] RBAC: Adding query filter params (role.id=f77ca21ddc4e463e98fbd3e9e7da7512, user.id=9b912d49c8a84405aa2cc61938024eb6, scope.system=all) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:56.922993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:56.925706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:56.933682 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 152/311] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Jul 25 05:40:56 2022] GET /identity/v3/role_assignments?user.id=9b912d49c8a84405aa2cc61938024eb6&scope.system=all&role.id=f77ca21ddc4e463e98fbd3e9e7da7512 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:56.961407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:56.980054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 160/312] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:57.006715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.066444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.129804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f58ed-c1a1-43b5-97ac-7a17d74565a9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.169429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465852c-e5d9-4bfa-ae85-89d5e76b4294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.170532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465852c-e5d9-4bfa-ae85-89d5e76b4294 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.170994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465852c-e5d9-4bfa-ae85-89d5e76b4294 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.171475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465852c-e5d9-4bfa-ae85-89d5e76b4294 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.182713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4465852c-e5d9-4bfa-ae85-89d5e76b4294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-4465852c-e5d9-4bfa-ae85-89d5e76b4294 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:57.190839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:57.191717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:57.192546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:57.194773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 153/313] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:40:57.201339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc88b01-342b-4b2c-8788-af7f465f4ee1 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.306579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc88b01-342b-4b2c-8788-af7f465f4ee1 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.341831 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.342507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.342786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.343055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.343803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:57.344105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.345508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.354810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 161/314] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:57.360352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465852c-e5d9-4bfa-ae85-89d5e76b4294 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.473375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465852c-e5d9-4bfa-ae85-89d5e76b4294 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.481482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.510055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9c126-2377-4eeb-abd5-5ecab1be84d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.511302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9c126-2377-4eeb-abd5-5ecab1be84d6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.511829 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9c126-2377-4eeb-abd5-5ecab1be84d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.512354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9c126-2377-4eeb-abd5-5ecab1be84d6 None admin] PATH_INFO: `/v3/users/c639cb172b23440f82e1fb1933470db5` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.513786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9c126-2377-4eeb-abd5-5ecab1be84d6 None admin] RBAC: Authorizing `identity:get_user(user_id=c639cb172b23440f82e1fb1933470db5)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.515428 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9c126-2377-4eeb-abd5-5ecab1be84d6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.517632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 154/315] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/users/c639cb172b23440f82e1fb1933470db5 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:57.526145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.584139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.631950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ae941-2f2a-4c92-bff1-da110aeac858 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.667846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baaa4e-20b1-4585-8ae7-e98a1755e788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.668531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baaa4e-20b1-4585-8ae7-e98a1755e788 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.668814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baaa4e-20b1-4585-8ae7-e98a1755e788 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.669087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baaa4e-20b1-4585-8ae7-e98a1755e788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.739838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baaa4e-20b1-4585-8ae7-e98a1755e788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.741333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baaa4e-20b1-4585-8ae7-e98a1755e788 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:57.800360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:57.847936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 162/316] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:40:57.853699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9c126-2377-4eeb-abd5-5ecab1be84d6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:57.958803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9c126-2377-4eeb-abd5-5ecab1be84d6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:57.994225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf77038-4a68-4265-a13e-5f56c1114ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:57.995092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf77038-4a68-4265-a13e-5f56c1114ab2 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:57.995375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf77038-4a68-4265-a13e-5f56c1114ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:57.995651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf77038-4a68-4265-a13e-5f56c1114ab2 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:57.996854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf77038-4a68-4265-a13e-5f56c1114ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:57.998297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf77038-4a68-4265-a13e-5f56c1114ab2 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.000297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 155/317] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:40:57 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:58.006113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baaa4e-20b1-4585-8ae7-e98a1755e788 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:58.112290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baaa4e-20b1-4585-8ae7-e98a1755e788 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.148260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d4443-c05f-406b-90bb-3c3b75056493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.148929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d4443-c05f-406b-90bb-3c3b75056493 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.149244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d4443-c05f-406b-90bb-3c3b75056493 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.149611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d4443-c05f-406b-90bb-3c3b75056493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.150721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d4443-c05f-406b-90bb-3c3b75056493 None admin] RBAC: Adding query filter params (role.id=f77ca21ddc4e463e98fbd3e9e7da7512, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7, user.id=c639cb172b23440f82e1fb1933470db5) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:58.151034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d4443-c05f-406b-90bb-3c3b75056493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:58.152520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d4443-c05f-406b-90bb-3c3b75056493 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.159015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 163/318] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:40:58 2022] GET /identity/v3/role_assignments?user.id=c639cb172b23440f82e1fb1933470db5&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=f77ca21ddc4e463e98fbd3e9e7da7512 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:58.381430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfeed1-f942-4c0c-bca8-88369991af07 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.381783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfeed1-f942-4c0c-bca8-88369991af07 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.381969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfeed1-f942-4c0c-bca8-88369991af07 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.383471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 156/319] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:58.388787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5f37-794f-479a-aca7-242ecd045fa8 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.389078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5f37-794f-479a-aca7-242ecd045fa8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.389352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e5f37-794f-479a-aca7-242ecd045fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.406752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-010e5f37-794f-479a-aca7-242ecd045fa8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:58.408283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-010e5f37-794f-479a-aca7-242ecd045fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:58.422527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:58.476292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 164/320] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:58.498804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfeed1-f942-4c0c-bca8-88369991af07 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:58.545631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:58.603089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfeed1-f942-4c0c-bca8-88369991af07 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.637435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf77fe9-fe92-472d-a83c-35b2438ae969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.638150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf77fe9-fe92-472d-a83c-35b2438ae969 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.638436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf77fe9-fe92-472d-a83c-35b2438ae969 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.638711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf77fe9-fe92-472d-a83c-35b2438ae969 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.639859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf77fe9-fe92-472d-a83c-35b2438ae969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:58.641333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf77fe9-fe92-472d-a83c-35b2438ae969 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.643125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 157/321] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:40:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:58.648619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5f37-794f-479a-aca7-242ecd045fa8 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:58.750184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e5f37-794f-479a-aca7-242ecd045fa8 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:58.792730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8c1b7-a97a-4a29-b022-734f8b57dbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:58.793406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c1b7-a97a-4a29-b022-734f8b57dbbf None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:58.793693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c1b7-a97a-4a29-b022-734f8b57dbbf None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:58.793970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8c1b7-a97a-4a29-b022-734f8b57dbbf None admin] PATH_INFO: `/v3/groups` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:58.794671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8c1b7-a97a-4a29-b022-734f8b57dbbf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:58.796150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8c1b7-a97a-4a29-b022-734f8b57dbbf None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:58.874253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 165/322] 10.222.0.40 () {60 vars in 1134 bytes} [Mon Jul 25 05:40:58 2022] POST /identity/v3/groups => generated 226 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:40:59.066126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c5042-31be-4b0a-8210-9c814e05f7a7 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.066629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c5042-31be-4b0a-8210-9c814e05f7a7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.066704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c5042-31be-4b0a-8210-9c814e05f7a7 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.068210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 158/323] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:59.074158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f1110-f6ad-4d1f-964f-7b7cd1abeeb6 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.074443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f1110-f6ad-4d1f-964f-7b7cd1abeeb6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.074718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f1110-f6ad-4d1f-964f-7b7cd1abeeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.091549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b86f1110-f6ad-4d1f-964f-7b7cd1abeeb6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:59.092757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b86f1110-f6ad-4d1f-964f-7b7cd1abeeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:59.095000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.111138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e957de-a9aa-4fcb-b0e1-b8521286a7ea None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.111412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e957de-a9aa-4fcb-b0e1-b8521286a7ea None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.111694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e957de-a9aa-4fcb-b0e1-b8521286a7ea None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.113173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 159/324] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:40:59.119022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60741c5-1e62-4af7-86f7-94e930dfc839 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.119420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60741c5-1e62-4af7-86f7-94e930dfc839 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.119705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60741c5-1e62-4af7-86f7-94e930dfc839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.141594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b60741c5-1e62-4af7-86f7-94e930dfc839 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:40:59.143124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b60741c5-1e62-4af7-86f7-94e930dfc839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:40:59.165667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 166/325] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:59.189258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f1110-f6ad-4d1f-964f-7b7cd1abeeb6 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.232204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 160/326] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:40:59.254299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60741c5-1e62-4af7-86f7-94e930dfc839 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.307118 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.331213 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f1110-f6ad-4d1f-964f-7b7cd1abeeb6 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.372562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2898b-616e-4aa7-a076-1c1e33c43bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.373378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2898b-616e-4aa7-a076-1c1e33c43bae None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.373664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2898b-616e-4aa7-a076-1c1e33c43bae None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.373939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2898b-616e-4aa7-a076-1c1e33c43bae None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.381385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.387081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2898b-616e-4aa7-a076-1c1e33c43bae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:59.391931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60741c5-1e62-4af7-86f7-94e930dfc839 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-79d2898b-616e-4aa7-a076-1c1e33c43bae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:59.395205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:59.396083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:40:59.396919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:40:59.397822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 167/327] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:59.403781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2898b-616e-4aa7-a076-1c1e33c43bae None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.442654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86d5ba-f859-4db3-803e-7ecc7dde8daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.444213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d5ba-f859-4db3-803e-7ecc7dde8daa None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.444891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d5ba-f859-4db3-803e-7ecc7dde8daa None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.445311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86d5ba-f859-4db3-803e-7ecc7dde8daa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.460555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f86d5ba-f859-4db3-803e-7ecc7dde8daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-3f86d5ba-f859-4db3-803e-7ecc7dde8daa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:40:59.471969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:40:59.473112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:40:59.474090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:40:59.480170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 161/328] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:40:59.482800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86d5ba-f859-4db3-803e-7ecc7dde8daa None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.561488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2898b-616e-4aa7-a076-1c1e33c43bae None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.597901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.604107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f86d5ba-f859-4db3-803e-7ecc7dde8daa None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.613817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.614706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.614998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.615279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.616086 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:59.616451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.617870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.630703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 168/329] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:59.640137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.662433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.663308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.663415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.663646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.664487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:59.665396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.666935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.686519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 162/330] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:59.691969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.725183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.781049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a50b572-279a-4160-ae2c-4d8aa9db4bc6 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.822313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e01a00-d9c2-4c95-9146-5925fc93a1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.823837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e01a00-d9c2-4c95-9146-5925fc93a1e8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.824351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e01a00-d9c2-4c95-9146-5925fc93a1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.824910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e01a00-d9c2-4c95-9146-5925fc93a1e8 None admin] PATH_INFO: `/v3/users/a0529e390b764b7bb6c1362f20af5978` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.826485 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e01a00-d9c2-4c95-9146-5925fc93a1e8 None admin] RBAC: Authorizing `identity:get_user(user_id=a0529e390b764b7bb6c1362f20af5978)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.828188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e01a00-d9c2-4c95-9146-5925fc93a1e8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.828531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cbde8-2640-4dd5-a136-af38db660c70 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.830822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 169/331] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/users/a0529e390b764b7bb6c1362f20af5978 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:40:59.836216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e01a00-d9c2-4c95-9146-5925fc93a1e8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.866171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bf7b4-28a7-44db-b62e-c6243f1fd4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.866987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bf7b4-28a7-44db-b62e-c6243f1fd4eb None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.867271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bf7b4-28a7-44db-b62e-c6243f1fd4eb None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.867548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bf7b4-28a7-44db-b62e-c6243f1fd4eb None admin] PATH_INFO: `/v3/users/9b912d49c8a84405aa2cc61938024eb6` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.870241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bf7b4-28a7-44db-b62e-c6243f1fd4eb None admin] RBAC: Authorizing `identity:get_user(user_id=9b912d49c8a84405aa2cc61938024eb6)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:40:59.871669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2bf7b4-28a7-44db-b62e-c6243f1fd4eb None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:40:59.873358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 163/332] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/users/9b912d49c8a84405aa2cc61938024eb6 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:40:59.879456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bf7b4-28a7-44db-b62e-c6243f1fd4eb None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:40:59.912839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.956640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e01a00-d9c2-4c95-9146-5925fc93a1e8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.986852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:40:59.989863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2bf7b4-28a7-44db-b62e-c6243f1fd4eb None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:40:59.996367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc094892-974f-4617-b428-8ea7e5711ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:40:59.997272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094892-974f-4617-b428-8ea7e5711ba5 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:40:59.997718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094892-974f-4617-b428-8ea7e5711ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:40:59.998157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094892-974f-4617-b428-8ea7e5711ba5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:40:59.999374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc094892-974f-4617-b428-8ea7e5711ba5 None admin] RBAC: Adding query filter params (role.id=45ad0499b0b54c309464a2ebb8a3e5d0, user.id=a0529e390b764b7bb6c1362f20af5978, scope.system=all) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:40:59.999866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc094892-974f-4617-b428-8ea7e5711ba5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:00.001671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc094892-974f-4617-b428-8ea7e5711ba5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.015998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 170/333] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Jul 25 05:40:59 2022] GET /identity/v3/role_assignments?user.id=a0529e390b764b7bb6c1362f20af5978&scope.system=all&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:00.028463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca7cc-eb98-483a-8fc7-45845eaffa4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:00.029368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca7cc-eb98-483a-8fc7-45845eaffa4e None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:00.029652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca7cc-eb98-483a-8fc7-45845eaffa4e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:00.029930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca7cc-eb98-483a-8fc7-45845eaffa4e None admin] PATH_INFO: `/v3/system/users/9b912d49c8a84405aa2cc61938024eb6/roles/f77ca21ddc4e463e98fbd3e9e7da7512` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:00.033001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ca7cc-eb98-483a-8fc7-45845eaffa4e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9b912d49c8a84405aa2cc61938024eb6, role_id=f77ca21ddc4e463e98fbd3e9e7da7512)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:00.034739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ca7cc-eb98-483a-8fc7-45845eaffa4e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:00.108045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 164/334] 10.222.0.40 () {58 vars in 1334 bytes} [Mon Jul 25 05:40:59 2022] PUT /identity/v3/system/users/9b912d49c8a84405aa2cc61938024eb6/roles/f77ca21ddc4e463e98fbd3e9e7da7512 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:41:01.836363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f33c-1ed3-4e06-a5c0-8a90a96645ab None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.838011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f33c-1ed3-4e06-a5c0-8a90a96645ab None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.838520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f33c-1ed3-4e06-a5c0-8a90a96645ab None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.840430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 171/335] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:01.845678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7054-ddf4-4547-a1eb-04771a975175 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:01.846104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7054-ddf4-4547-a1eb-04771a975175 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:01.846224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d7054-ddf4-4547-a1eb-04771a975175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:01.873126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2c5d7054-ddf4-4547-a1eb-04771a975175 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:01.874667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2c5d7054-ddf4-4547-a1eb-04771a975175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:01.955249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 165/336] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:01.999414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f33c-1ed3-4e06-a5c0-8a90a96645ab None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.150479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f33c-1ed3-4e06-a5c0-8a90a96645ab None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.186072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366208-e47f-4e47-92ba-48d829ac60dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.186884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366208-e47f-4e47-92ba-48d829ac60dc None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.187171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366208-e47f-4e47-92ba-48d829ac60dc None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.187448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366208-e47f-4e47-92ba-48d829ac60dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.207097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46366208-e47f-4e47-92ba-48d829ac60dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-46366208-e47f-4e47-92ba-48d829ac60dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:02.215055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:02.215910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:02.216729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 172/337] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:02.222789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d7054-ddf4-4547-a1eb-04771a975175 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.271857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.327296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7686b5-d5ee-4453-9f93-0382627d5095 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.327587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7686b5-d5ee-4453-9f93-0382627d5095 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.328126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7686b5-d5ee-4453-9f93-0382627d5095 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.330761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 173/338] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:02.339584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77530421-a1b1-4190-8499-592457d9296c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.340133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77530421-a1b1-4190-8499-592457d9296c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.343324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77530421-a1b1-4190-8499-592457d9296c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.373152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-77530421-a1b1-4190-8499-592457d9296c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:02.374837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-77530421-a1b1-4190-8499-592457d9296c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:02.398395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d7054-ddf4-4547-a1eb-04771a975175 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.475646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.496878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71f51d-6b20-49ac-b354-46df550fda44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.497571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f51d-6b20-49ac-b354-46df550fda44 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.497855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f51d-6b20-49ac-b354-46df550fda44 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.498127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71f51d-6b20-49ac-b354-46df550fda44 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.498881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71f51d-6b20-49ac-b354-46df550fda44 None admin] RBAC: Adding query filter params (name=member) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:02.499190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71f51d-6b20-49ac-b354-46df550fda44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.500663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71f51d-6b20-49ac-b354-46df550fda44 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.515677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 166/339] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:02.522719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feff41-9666-4396-bcbf-efbdb97eeb09 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.523006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feff41-9666-4396-bcbf-efbdb97eeb09 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.523294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feff41-9666-4396-bcbf-efbdb97eeb09 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.525440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 167/340] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:02.526790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 174/341] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:02.528477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79feff41-9666-4396-bcbf-efbdb97eeb09 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.544010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3c21-b242-4a00-b81c-7c68b6f86ee4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.544359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3c21-b242-4a00-b81c-7c68b6f86ee4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.544646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d3c21-b242-4a00-b81c-7c68b6f86ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.568128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d83d3c21-b242-4a00-b81c-7c68b6f86ee4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:02.569411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d83d3c21-b242-4a00-b81c-7c68b6f86ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:02.643322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 175/342] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:02.643635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d3c21-b242-4a00-b81c-7c68b6f86ee4 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.670881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.688092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79feff41-9666-4396-bcbf-efbdb97eeb09 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.730594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06f6d-b468-4383-a166-fcaaedba4e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.731613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06f6d-b468-4383-a166-fcaaedba4e6d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.732096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06f6d-b468-4383-a166-fcaaedba4e6d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.732668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e06f6d-b468-4383-a166-fcaaedba4e6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.755547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83d3c21-b242-4a00-b81c-7c68b6f86ee4 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.764877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.795851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f260aa-0479-46be-9938-4d610dc92fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.796781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f260aa-0479-46be-9938-4d610dc92fb8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.797279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f260aa-0479-46be-9938-4d610dc92fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.797731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f260aa-0479-46be-9938-4d610dc92fb8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:02.810485 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f260aa-0479-46be-9938-4d610dc92fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:02.810660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e06f6d-b468-4383-a166-fcaaedba4e6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:02.817258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e06f6d-b468-4383-a166-fcaaedba4e6d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-31f260aa-0479-46be-9938-4d610dc92fb8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:02.819777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:02.820694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:02.822190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:02.825196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 176/343] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:02.826305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f260aa-0479-46be-9938-4d610dc92fb8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.915531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:02.936860 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 168/344] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:02.937838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06f6d-b468-4383-a166-fcaaedba4e6d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:02.948663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f260aa-0479-46be-9938-4d610dc92fb8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:02.987221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49005c-f7aa-4a77-ad79-a7d08ab3a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:02.988353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49005c-f7aa-4a77-ad79-a7d08ab3a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:02.988842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49005c-f7aa-4a77-ad79-a7d08ab3a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:02.989375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49005c-f7aa-4a77-ad79-a7d08ab3a5b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.001006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49005c-f7aa-4a77-ad79-a7d08ab3a5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-ee49005c-f7aa-4a77-ad79-a7d08ab3a5b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:03.010698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:03.011620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:03.012534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:03.015183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 177/345] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:03.016195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49005c-f7aa-4a77-ad79-a7d08ab3a5b0 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.058837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e06f6d-b468-4383-a166-fcaaedba4e6d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.072816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.097859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.098354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.098706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.099055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.099945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] RBAC: Adding query filter params (name=member) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:03.100343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.101817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.112228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 169/346] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:41:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:03.113231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.133089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49005c-f7aa-4a77-ad79-a7d08ab3a5b0 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.167222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.174424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b27af2-ad81-49b2-ab7a-f498a875f94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.175553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b27af2-ad81-49b2-ab7a-f498a875f94f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.176057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b27af2-ad81-49b2-ab7a-f498a875f94f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.176660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b27af2-ad81-49b2-ab7a-f498a875f94f None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.186932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b27af2-ad81-49b2-ab7a-f498a875f94f None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.186932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b27af2-ad81-49b2-ab7a-f498a875f94f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.186932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 178/347] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:03.186932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b27af2-ad81-49b2-ab7a-f498a875f94f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.223257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7185143-8f63-4efc-8d9d-39f6b7bcf878 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.263287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.263880 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.264178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.264450 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.265270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:03.265570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.267112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.277201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 170/348] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:03.282519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.300484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.308376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b27af2-ad81-49b2-ab7a-f498a875f94f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.389137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957c41-5656-491e-bc90-51a4d61fd457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.389808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957c41-5656-491e-bc90-51a4d61fd457 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.390101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957c41-5656-491e-bc90-51a4d61fd457 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.390379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb957c41-5656-491e-bc90-51a4d61fd457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.395596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8fad66-a04e-4da1-985b-a2a59915acbe None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.409577 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.435634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903585f-9041-439c-9b7f-4488091c11a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.436540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903585f-9041-439c-9b7f-4488091c11a6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.436819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903585f-9041-439c-9b7f-4488091c11a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.437245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903585f-9041-439c-9b7f-4488091c11a6 None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.444225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903585f-9041-439c-9b7f-4488091c11a6 None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.445779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903585f-9041-439c-9b7f-4488091c11a6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.449264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 171/349] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:03.450296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903585f-9041-439c-9b7f-4488091c11a6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.494452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb957c41-5656-491e-bc90-51a4d61fd457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.496690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb957c41-5656-491e-bc90-51a4d61fd457 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.600420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903585f-9041-439c-9b7f-4488091c11a6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.612003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.621537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 179/350] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:03.622381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957c41-5656-491e-bc90-51a4d61fd457 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.645296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bce227-2c0b-4472-8a1b-30cf494d7b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.645958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bce227-2c0b-4472-8a1b-30cf494d7b2e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.646239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bce227-2c0b-4472-8a1b-30cf494d7b2e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.646515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bce227-2c0b-4472-8a1b-30cf494d7b2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.695614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.725080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bce227-2c0b-4472-8a1b-30cf494d7b2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.726775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bce227-2c0b-4472-8a1b-30cf494d7b2e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.745677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb957c41-5656-491e-bc90-51a4d61fd457 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.792457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367dcab-b131-4f50-ac62-b655e5469e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:03.793392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367dcab-b131-4f50-ac62-b655e5469e45 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.793837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367dcab-b131-4f50-ac62-b655e5469e45 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.794116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367dcab-b131-4f50-ac62-b655e5469e45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.795349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367dcab-b131-4f50-ac62-b655e5469e45 None admin] RBAC: Adding query filter params (group.id=2f4fb549e19b4d239032ccc5654ba389, role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:03.795349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367dcab-b131-4f50-ac62-b655e5469e45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:03.796766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8367dcab-b131-4f50-ac62-b655e5469e45 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:03.802502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 180/351] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/role_assignments?group.id=2f4fb549e19b4d239032ccc5654ba389&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:03.803993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270e818-8500-4c16-83e8-e727353c208d None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.804317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270e818-8500-4c16-83e8-e727353c208d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.804599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270e818-8500-4c16-83e8-e727353c208d None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.806482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 181/352] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:03.807708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73e873-0280-4db4-85ba-09dd4b2234c8 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.808063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73e873-0280-4db4-85ba-09dd4b2234c8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.808396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73e873-0280-4db4-85ba-09dd4b2234c8 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.809724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 182/353] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:03.810477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73e873-0280-4db4-85ba-09dd4b2234c8 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:03.871388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 172/354] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:03.873414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9baa-77db-4799-8697-eef6caa79bc1 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.873937 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9baa-77db-4799-8697-eef6caa79bc1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.874395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d9baa-77db-4799-8697-eef6caa79bc1 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.881308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 173/355] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:03.882807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a22a7c-46b9-4b22-96f6-25cce7d2d857 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:03.883103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a22a7c-46b9-4b22-96f6-25cce7d2d857 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:03.883986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a22a7c-46b9-4b22-96f6-25cce7d2d857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:03.906128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:03.912779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-47a22a7c-46b9-4b22-96f6-25cce7d2d857 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:03.912779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-47a22a7c-46b9-4b22-96f6-25cce7d2d857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:03.962764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73e873-0280-4db4-85ba-09dd4b2234c8 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:03.999330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 174/356] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:04.000738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc749b-48f3-4fcf-846a-9ad3c3dd62ba None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.001034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc749b-48f3-4fcf-846a-9ad3c3dd62ba None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.001312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc749b-48f3-4fcf-846a-9ad3c3dd62ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.006479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8138c54-9aec-4a9f-8cea-0c984e38d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.006987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.008919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138c54-9aec-4a9f-8cea-0c984e38d67b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.009423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138c54-9aec-4a9f-8cea-0c984e38d67b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.009773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8138c54-9aec-4a9f-8cea-0c984e38d67b None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.011750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8138c54-9aec-4a9f-8cea-0c984e38d67b None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:04.012779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8138c54-9aec-4a9f-8cea-0c984e38d67b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:04.014667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 183/357] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:03 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:04.015542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8138c54-9aec-4a9f-8cea-0c984e38d67b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.021144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-40dc749b-48f3-4fcf-846a-9ad3c3dd62ba None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:04.022676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-40dc749b-48f3-4fcf-846a-9ad3c3dd62ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:04.108532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 175/358] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:04.110000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299e0c9-4c4d-48da-949c-a3da428ecb6a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.110293 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299e0c9-4c4d-48da-949c-a3da428ecb6a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.110570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299e0c9-4c4d-48da-949c-a3da428ecb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.136913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a299e0c9-4c4d-48da-949c-a3da428ecb6a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:04.138821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a299e0c9-4c4d-48da-949c-a3da428ecb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:04.153473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8138c54-9aec-4a9f-8cea-0c984e38d67b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.186036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.196287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d358fa-5792-4033-bc5b-fbe96ef8533a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.200172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d358fa-5792-4033-bc5b-fbe96ef8533a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.200172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d358fa-5792-4033-bc5b-fbe96ef8533a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.200172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d358fa-5792-4033-bc5b-fbe96ef8533a None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.203712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d358fa-5792-4033-bc5b-fbe96ef8533a None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:04.203712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d358fa-5792-4033-bc5b-fbe96ef8533a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:04.206980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 184/359] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:04.206980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d358fa-5792-4033-bc5b-fbe96ef8533a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.241945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 176/360] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:04.243003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299e0c9-4c4d-48da-949c-a3da428ecb6a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.319291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d358fa-5792-4033-bc5b-fbe96ef8533a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.321241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.361161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4342059-626f-46f3-a019-336af2d06fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.362046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342059-626f-46f3-a019-336af2d06fcf None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.362331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342059-626f-46f3-a019-336af2d06fcf None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.362766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342059-626f-46f3-a019-336af2d06fcf None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.364348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4342059-626f-46f3-a019-336af2d06fcf None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:04.365825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4342059-626f-46f3-a019-336af2d06fcf None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:04.367478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 185/361] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:04.368835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4342059-626f-46f3-a019-336af2d06fcf None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.369877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299e0c9-4c4d-48da-949c-a3da428ecb6a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.411706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769be187-7114-4518-94b3-6eaae23ccfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.412613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be187-7114-4518-94b3-6eaae23ccfce None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.413064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be187-7114-4518-94b3-6eaae23ccfce None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.413352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769be187-7114-4518-94b3-6eaae23ccfce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.428420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769be187-7114-4518-94b3-6eaae23ccfce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-769be187-7114-4518-94b3-6eaae23ccfce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:04.439151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:04.440039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:04.440859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 177/362] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:04.442055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769be187-7114-4518-94b3-6eaae23ccfce None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.460095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.510835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4342059-626f-46f3-a019-336af2d06fcf None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.563765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ee815-ac06-49f4-9936-947ceaca61f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.564613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee815-ac06-49f4-9936-947ceaca61f8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.564899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee815-ac06-49f4-9936-947ceaca61f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.565176 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ee815-ac06-49f4-9936-947ceaca61f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.583857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ee815-ac06-49f4-9936-947ceaca61f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:04.592895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769be187-7114-4518-94b3-6eaae23ccfce None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-c93ee815-ac06-49f4-9936-947ceaca61f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:04.593148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:04.594005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:04.594781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 186/363] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:04.596039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ee815-ac06-49f4-9936-947ceaca61f8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.625561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.641781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a66dbe-31bc-48f6-a719-dfca34c7136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.642473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a66dbe-31bc-48f6-a719-dfca34c7136f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.642757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a66dbe-31bc-48f6-a719-dfca34c7136f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.643035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a66dbe-31bc-48f6-a719-dfca34c7136f None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.645534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a66dbe-31bc-48f6-a719-dfca34c7136f None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:04.645862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a66dbe-31bc-48f6-a719-dfca34c7136f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:04.647516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 178/364] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:04.655317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a66dbe-31bc-48f6-a719-dfca34c7136f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.729862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ee815-ac06-49f4-9936-947ceaca61f8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.761195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.773780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a66dbe-31bc-48f6-a719-dfca34c7136f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.776446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32df8f-12ec-47cc-a9af-1c241f803958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.777346 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32df8f-12ec-47cc-a9af-1c241f803958 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.778530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32df8f-12ec-47cc-a9af-1c241f803958 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.778530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32df8f-12ec-47cc-a9af-1c241f803958 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.793848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32df8f-12ec-47cc-a9af-1c241f803958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-bd32df8f-12ec-47cc-a9af-1c241f803958 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:04.803812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:04.804838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 25 05:41:04.805620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:04.806227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 187/365] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:04.806966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32df8f-12ec-47cc-a9af-1c241f803958 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.821638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.822713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.822969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.823247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.824091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] RBAC: Adding query filter params (group.id=2f4fb549e19b4d239032ccc5654ba389, role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:04.824449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:04.825951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:04.834721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 179/366] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/role_assignments?group.id=2f4fb549e19b4d239032ccc5654ba389&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:04.835560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:04.907947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:04.927253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32df8f-12ec-47cc-a9af-1c241f803958 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.963936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5bbc3-40cd-4bbb-9e67-fd952d38c10a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:04.966989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:04.967665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:04.967952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:04.969152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:04.969152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:04.969332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:04.970768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:04.981161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 188/367] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:04.982169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.018090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.018784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.019073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.020322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.020322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:05.020478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.021842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.035474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 180/368] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:05.036804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.037584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.106414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c125df-4548-415e-a916-4e191d0c84ef None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.159960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f541812-7cfb-4271-a45b-c406233c1791 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.162709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.164531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.165401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.165690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.166561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] RBAC: Adding query filter params (group.id=2f4fb549e19b4d239032ccc5654ba389, role.id=20a552366bf14792b05decae218a79ee, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:05.166878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.168373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.184154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 189/369] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:04 2022] GET /identity/v3/role_assignments?group.id=2f4fb549e19b4d239032ccc5654ba389&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=20a552366bf14792b05decae218a79ee => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:05.184154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.187298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.230764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.233825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.234044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.234324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.235111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:05.235423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.237370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.257124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 181/370] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:05.257824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.389624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b16af-ea18-40b2-9a3b-9081b2094b10 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.403638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.432652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c6041-c4db-41ed-802f-24e71c709b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.433710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6041-c4db-41ed-802f-24e71c709b52 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.434208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6041-c4db-41ed-802f-24e71c709b52 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.434680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c6041-c4db-41ed-802f-24e71c709b52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.497096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760ea81-8809-443f-a181-875eb9418f31 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.544710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c6041-c4db-41ed-802f-24e71c709b52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.547179 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c6041-c4db-41ed-802f-24e71c709b52 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.566760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03aef-1613-4102-9e5e-7535c8a5dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.567791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03aef-1613-4102-9e5e-7535c8a5dc01 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.568308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03aef-1613-4102-9e5e-7535c8a5dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.568786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03aef-1613-4102-9e5e-7535c8a5dc01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.679724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03aef-1613-4102-9e5e-7535c8a5dc01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.682047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03aef-1613-4102-9e5e-7535c8a5dc01 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.690979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 190/371] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:05.692095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c6041-c4db-41ed-802f-24e71c709b52 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.700552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.704242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.802004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c6041-c4db-41ed-802f-24e71c709b52 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.839138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 182/372] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:05.840009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03aef-1613-4102-9e5e-7535c8a5dc01 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.841129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75206d-7aef-49fd-8649-4e887c39f310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:05.842245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75206d-7aef-49fd-8649-4e887c39f310 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:05.842724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75206d-7aef-49fd-8649-4e887c39f310 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:05.843275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75206d-7aef-49fd-8649-4e887c39f310 None admin] PATH_INFO: `/v3/users/9b912d49c8a84405aa2cc61938024eb6` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:05.844743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75206d-7aef-49fd-8649-4e887c39f310 None admin] RBAC: Authorizing `identity:get_user(user_id=9b912d49c8a84405aa2cc61938024eb6)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:05.846396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a75206d-7aef-49fd-8649-4e887c39f310 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:05.848310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 191/373] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/users/9b912d49c8a84405aa2cc61938024eb6 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:05.849256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75206d-7aef-49fd-8649-4e887c39f310 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:05.952140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.967684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03aef-1613-4102-9e5e-7535c8a5dc01 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:05.974360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:05.999641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75206d-7aef-49fd-8649-4e887c39f310 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.004646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57a1d4-1b85-4d06-8d0d-b1105ac214e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.005365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a1d4-1b85-4d06-8d0d-b1105ac214e0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.005729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a1d4-1b85-4d06-8d0d-b1105ac214e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.006103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a1d4-1b85-4d06-8d0d-b1105ac214e0 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.007901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57a1d4-1b85-4d06-8d0d-b1105ac214e0 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.010305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57a1d4-1b85-4d06-8d0d-b1105ac214e0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.016459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 183/374] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.017483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57a1d4-1b85-4d06-8d0d-b1105ac214e0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.040150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d9d69-77a0-4254-bb0b-291545861c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.041299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d9d69-77a0-4254-bb0b-291545861c99 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.041780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d9d69-77a0-4254-bb0b-291545861c99 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.042229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475d9d69-77a0-4254-bb0b-291545861c99 None admin] PATH_INFO: `/v3/projects/1d6ac2b8c3194197aedc989787229928` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.044786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d9d69-77a0-4254-bb0b-291545861c99 None admin] RBAC: Authorizing `identity:get_project(project_id=1d6ac2b8c3194197aedc989787229928)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.046421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475d9d69-77a0-4254-bb0b-291545861c99 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.048642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 192/375] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:05 2022] GET /identity/v3/projects/1d6ac2b8c3194197aedc989787229928 => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.049787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d9d69-77a0-4254-bb0b-291545861c99 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.133717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57a1d4-1b85-4d06-8d0d-b1105ac214e0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.158391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475d9d69-77a0-4254-bb0b-291545861c99 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.172003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.172702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.172990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.173270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.174112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] RBAC: Adding query filter params (role.id=f77ca21ddc4e463e98fbd3e9e7da7512, user.id=9b912d49c8a84405aa2cc61938024eb6, scope.system=all) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:06.174427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.176316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.190114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 184/376] 10.222.0.40 () {56 vars in 1308 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/role_assignments?user.id=9b912d49c8a84405aa2cc61938024eb6&scope.system=all&role.id=f77ca21ddc4e463e98fbd3e9e7da7512 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.190964 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.206916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.212990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f01e6f-1ad9-4954-ab82-bb78fd9ee926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.214256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f01e6f-1ad9-4954-ab82-bb78fd9ee926 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.214760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f01e6f-1ad9-4954-ab82-bb78fd9ee926 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.215234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f01e6f-1ad9-4954-ab82-bb78fd9ee926 None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.216745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f01e6f-1ad9-4954-ab82-bb78fd9ee926 None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.218400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f01e6f-1ad9-4954-ab82-bb78fd9ee926 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.220572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 193/377] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.222066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1061b-e1ac-4903-8967-1117a7cd00a6 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.222552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1061b-e1ac-4903-8967-1117a7cd00a6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.223025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f1061b-e1ac-4903-8967-1117a7cd00a6 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.224706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 194/378] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:06.231129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23974f4-28ab-4fbf-a915-4917eb76e26e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.231751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23974f4-28ab-4fbf-a915-4917eb76e26e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.232266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23974f4-28ab-4fbf-a915-4917eb76e26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.250759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c23974f4-28ab-4fbf-a915-4917eb76e26e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:06.252321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c23974f4-28ab-4fbf-a915-4917eb76e26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:06.261863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.361374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b50850-61d8-4d0b-80ed-da02128b94cf None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.365840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 195/379] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:06.366697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23974f4-28ab-4fbf-a915-4917eb76e26e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.411456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029900a1-119e-4076-be18-73669a13886d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.412779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029900a1-119e-4076-be18-73669a13886d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.413859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029900a1-119e-4076-be18-73669a13886d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.414472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029900a1-119e-4076-be18-73669a13886d None admin] PATH_INFO: `/v3/groups/9b471f0a21a04e4ba8e39be6f49c1810` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.422354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029900a1-119e-4076-be18-73669a13886d None admin] RBAC: Authorizing `identity:get_group(group_id=9b471f0a21a04e4ba8e39be6f49c1810)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.424199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029900a1-119e-4076-be18-73669a13886d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.426543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 185/380] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/groups/9b471f0a21a04e4ba8e39be6f49c1810 => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.427606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029900a1-119e-4076-be18-73669a13886d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.525352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.562643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029900a1-119e-4076-be18-73669a13886d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.592839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23974f4-28ab-4fbf-a915-4917eb76e26e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.604534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0134c7a-8f61-4b6d-a26d-2f5205104a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.605649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0134c7a-8f61-4b6d-a26d-2f5205104a69 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.606157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0134c7a-8f61-4b6d-a26d-2f5205104a69 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.606639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0134c7a-8f61-4b6d-a26d-2f5205104a69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.618212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0134c7a-8f61-4b6d-a26d-2f5205104a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b0134c7a-8f61-4b6d-a26d-2f5205104a69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:06.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:06.627404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:06.628203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:06.631006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 186/381] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:06.632285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0134c7a-8f61-4b6d-a26d-2f5205104a69 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.632874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.633837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.634125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.634403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.635248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] RBAC: Adding query filter params (group.id=2f4fb549e19b4d239032ccc5654ba389, role.id=20a552366bf14792b05decae218a79ee, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:06.635593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.637340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.643517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 196/382] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/role_assignments?group.id=2f4fb549e19b4d239032ccc5654ba389&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=20a552366bf14792b05decae218a79ee => generated 260 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.648456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.661732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.747064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0134c7a-8f61-4b6d-a26d-2f5205104a69 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.786451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:06.792261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.793061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cfa840-2a28-4778-9570-cf65de2560bc None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:06.793490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.793972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.794563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.795738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] RBAC: Adding query filter params (group.id=9b471f0a21a04e4ba8e39be6f49c1810, role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=1d6ac2b8c3194197aedc989787229928) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:06.796294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.797974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.805134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 187/383] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/role_assignments?group.id=9b471f0a21a04e4ba8e39be6f49c1810&scope.project.id=1d6ac2b8c3194197aedc989787229928&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.834931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:06.835629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:06.835914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:06.836414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:06.837813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:06.838128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:06.839644 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:06.850764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 197/384] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:06.856877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:06.996165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb749d66-de6e-4749-a217-05cbec0de240 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.032411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d36c36-e7a9-4fb5-8cfc-4825326d3b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.033543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36c36-e7a9-4fb5-8cfc-4825326d3b43 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.033971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36c36-e7a9-4fb5-8cfc-4825326d3b43 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.034392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d36c36-e7a9-4fb5-8cfc-4825326d3b43 None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.035714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d36c36-e7a9-4fb5-8cfc-4825326d3b43 None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.037318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d36c36-e7a9-4fb5-8cfc-4825326d3b43 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.039946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 188/385] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:06 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:07.045874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.125770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.163487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939ed8f-96cc-474e-8e3b-9cf0b786b90a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.201156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbc8a7-c905-436c-9bd3-96dd9ae3c8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.202041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbc8a7-c905-436c-9bd3-96dd9ae3c8d0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.202326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbc8a7-c905-436c-9bd3-96dd9ae3c8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.202605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbc8a7-c905-436c-9bd3-96dd9ae3c8d0 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.203895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbc8a7-c905-436c-9bd3-96dd9ae3c8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbc8a7-c905-436c-9bd3-96dd9ae3c8d0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.207759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 198/386] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:07.213849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d36c36-e7a9-4fb5-8cfc-4825326d3b43 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.307712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.352740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d36c36-e7a9-4fb5-8cfc-4825326d3b43 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.403573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e8c3b-925d-49c9-8909-38873eb52d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.404503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e8c3b-925d-49c9-8909-38873eb52d39 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.404788 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e8c3b-925d-49c9-8909-38873eb52d39 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.405065 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e8c3b-925d-49c9-8909-38873eb52d39 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/groups/2f4fb549e19b4d239032ccc5654ba389/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.407052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e8c3b-925d-49c9-8909-38873eb52d39 None admin] RBAC: Authorizing `identity:create_grant(project_id=39a8b4919bb540a7a257d26089c8b3bd, group_id=2f4fb549e19b4d239032ccc5654ba389, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:07.408544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e8c3b-925d-49c9-8909-38873eb52d39 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:07.496947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 189/387] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 25 05:41:07 2022] PUT /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/groups/2f4fb549e19b4d239032ccc5654ba389/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:07.522616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b57c-7b2b-4e9e-88c3-02ad01316723 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.523276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b57c-7b2b-4e9e-88c3-02ad01316723 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.523904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b57c-7b2b-4e9e-88c3-02ad01316723 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.525756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 199/388] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:07.532893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417b94-8491-4587-847d-7a7d2a1f6d2c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.533231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417b94-8491-4587-847d-7a7d2a1f6d2c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.533467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88417b94-8491-4587-847d-7a7d2a1f6d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.553497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-88417b94-8491-4587-847d-7a7d2a1f6d2c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:07.554718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-88417b94-8491-4587-847d-7a7d2a1f6d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:07.637802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 190/389] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:07.681868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52b57c-7b2b-4e9e-88c3-02ad01316723 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.816647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52b57c-7b2b-4e9e-88c3-02ad01316723 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:07.857861 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819c587-f704-4f0c-9b2f-9d546ea53d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:07.858668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819c587-f704-4f0c-9b2f-9d546ea53d19 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:07.858955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819c587-f704-4f0c-9b2f-9d546ea53d19 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:07.859237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819c587-f704-4f0c-9b2f-9d546ea53d19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:07.862093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:07.873274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819c587-f704-4f0c-9b2f-9d546ea53d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-3819c587-f704-4f0c-9b2f-9d546ea53d19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:07.881252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:07.882200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:07.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 200/390] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:07.889349 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417b94-8491-4587-847d-7a7d2a1f6d2c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:07.962507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.020696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88417b94-8491-4587-847d-7a7d2a1f6d2c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.068833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83112a3-50ad-4285-bc62-9f680e969864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.069523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83112a3-50ad-4285-bc62-9f680e969864 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.069808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83112a3-50ad-4285-bc62-9f680e969864 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.070089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83112a3-50ad-4285-bc62-9f680e969864 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.071703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83112a3-50ad-4285-bc62-9f680e969864 None admin] RBAC: Adding query filter params (name=member) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:08.072025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83112a3-50ad-4285-bc62-9f680e969864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.073724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83112a3-50ad-4285-bc62-9f680e969864 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.082504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 191/391] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:41:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:08.088710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819c587-f704-4f0c-9b2f-9d546ea53d19 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.170209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1484-715e-420f-842b-7a4cdbf872a7 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.170543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1484-715e-420f-842b-7a4cdbf872a7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.170781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa1484-715e-420f-842b-7a4cdbf872a7 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.172567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 192/392] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:08.177572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c94b94-baa9-4a83-bdc4-9da340c98d3f None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.177951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c94b94-baa9-4a83-bdc4-9da340c98d3f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.178235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c94b94-baa9-4a83-bdc4-9da340c98d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.204793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819c587-f704-4f0c-9b2f-9d546ea53d19 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.210294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-94c94b94-baa9-4a83-bdc4-9da340c98d3f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:08.212064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-94c94b94-baa9-4a83-bdc4-9da340c98d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:08.240537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6909e01-e06d-4f27-bfa4-fbe7d906dbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.241479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909e01-e06d-4f27-bfa4-fbe7d906dbf8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.241767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909e01-e06d-4f27-bfa4-fbe7d906dbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.242038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6909e01-e06d-4f27-bfa4-fbe7d906dbf8 None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.243410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6909e01-e06d-4f27-bfa4-fbe7d906dbf8 None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.244897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6909e01-e06d-4f27-bfa4-fbe7d906dbf8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.246848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 201/393] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:08 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:08.252227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6909e01-e06d-4f27-bfa4-fbe7d906dbf8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.352842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.373685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6909e01-e06d-4f27-bfa4-fbe7d906dbf8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.416075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937afa2-3620-407b-b943-a72124bf4213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.417241 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937afa2-3620-407b-b943-a72124bf4213 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.417589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937afa2-3620-407b-b943-a72124bf4213 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.417934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a937afa2-3620-407b-b943-a72124bf4213 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.419513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937afa2-3620-407b-b943-a72124bf4213 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.421233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a937afa2-3620-407b-b943-a72124bf4213 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.423325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 202/394] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:08 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:08.426243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 193/395] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:08.441694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937afa2-3620-407b-b943-a72124bf4213 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.460052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c94b94-baa9-4a83-bdc4-9da340c98d3f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.567461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a937afa2-3620-407b-b943-a72124bf4213 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.608565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.611020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:08.614090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d933-c995-4e48-a570-7aafb11186f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.614973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d933-c995-4e48-a570-7aafb11186f5 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.615258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d933-c995-4e48-a570-7aafb11186f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.615531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d933-c995-4e48-a570-7aafb11186f5 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/groups/2f4fb549e19b4d239032ccc5654ba389/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.619948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d933-c995-4e48-a570-7aafb11186f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a1c784d8b9e4949be326e17465c7aa7, group_id=2f4fb549e19b4d239032ccc5654ba389, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.624314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d933-c995-4e48-a570-7aafb11186f5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.682482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c94b94-baa9-4a83-bdc4-9da340c98d3f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.716093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 203/396] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 25 05:41:08 2022] PUT /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/groups/2f4fb549e19b4d239032ccc5654ba389/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:41:08.720840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6e7c6-fda2-4510-ab84-f3175a880930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.721679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6e7c6-fda2-4510-ab84-f3175a880930 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.722011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6e7c6-fda2-4510-ab84-f3175a880930 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.722572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6e7c6-fda2-4510-ab84-f3175a880930 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.735303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6e7c6-fda2-4510-ab84-f3175a880930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-5fc6e7c6-fda2-4510-ab84-f3175a880930 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:08.745120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:08.746322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:08.747111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 194/397] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:08.756614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d933-c995-4e48-a570-7aafb11186f5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:08.934504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d933-c995-4e48-a570-7aafb11186f5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:08.981928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:08.982612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:08.982894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:08.983171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:08.983932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:08.984290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:08.985678 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:08.999438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 204/398] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:09.005049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6e7c6-fda2-4510-ab84-f3175a880930 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:09.135829 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6e7c6-fda2-4510-ab84-f3175a880930 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:09.138407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:09.171812 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42797458-964b-4a66-bb75-fedf72977617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:09.172720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42797458-964b-4a66-bb75-fedf72977617 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:09.173004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42797458-964b-4a66-bb75-fedf72977617 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:09.173279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42797458-964b-4a66-bb75-fedf72977617 None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:09.174469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42797458-964b-4a66-bb75-fedf72977617 None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:09.175933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42797458-964b-4a66-bb75-fedf72977617 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:09.177855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 195/399] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:09 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:09.184251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:09.266956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:09.314284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b00136-bb92-40a9-8676-ffd33655a949 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:09.359050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a74e1-7b9a-4908-b7b2-30cefab6cfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:09.359973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a74e1-7b9a-4908-b7b2-30cefab6cfea None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:09.360288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a74e1-7b9a-4908-b7b2-30cefab6cfea None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:09.360565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a74e1-7b9a-4908-b7b2-30cefab6cfea None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:09.361801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a74e1-7b9a-4908-b7b2-30cefab6cfea None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:09.363251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a74e1-7b9a-4908-b7b2-30cefab6cfea None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:09.365012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 205/400] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:09 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:09.374758 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42797458-964b-4a66-bb75-fedf72977617 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:09.457177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978ebd2-9979-4cb7-847c-08bc836f5a12 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:09.457512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978ebd2-9979-4cb7-847c-08bc836f5a12 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:09.457740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978ebd2-9979-4cb7-847c-08bc836f5a12 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:09.459862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 206/401] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:09.467914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea778c2-d3a7-4c22-a442-b1e9e6e2d014 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:09.468234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea778c2-d3a7-4c22-a442-b1e9e6e2d014 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:09.468522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea778c2-d3a7-4c22-a442-b1e9e6e2d014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:09.496916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-5ea778c2-d3a7-4c22-a442-b1e9e6e2d014 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:09.499766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5ea778c2-d3a7-4c22-a442-b1e9e6e2d014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:09.554418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42797458-964b-4a66-bb75-fedf72977617 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:09.610340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094748c6-8784-4099-82f2-16b4efafe2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:09.611581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094748c6-8784-4099-82f2-16b4efafe2e1 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:09.612089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094748c6-8784-4099-82f2-16b4efafe2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:09.612656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094748c6-8784-4099-82f2-16b4efafe2e1 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/groups/2f4fb549e19b4d239032ccc5654ba389/roles/20a552366bf14792b05decae218a79ee` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:09.615931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094748c6-8784-4099-82f2-16b4efafe2e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=39a8b4919bb540a7a257d26089c8b3bd, group_id=2f4fb549e19b4d239032ccc5654ba389, role_id=20a552366bf14792b05decae218a79ee)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:09.617762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094748c6-8784-4099-82f2-16b4efafe2e1 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:09.639427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 207/402] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:09.665933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea778c2-d3a7-4c22-a442-b1e9e6e2d014 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:09.705330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 196/403] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 25 05:41:09 2022] PUT /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd/groups/2f4fb549e19b4d239032ccc5654ba389/roles/20a552366bf14792b05decae218a79ee => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:09.707212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c71385-20be-4d74-8ddc-945418ae6796 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:09.707714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c71385-20be-4d74-8ddc-945418ae6796 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:09.708344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c71385-20be-4d74-8ddc-945418ae6796 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:09.714460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 197/404] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:09.717071 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c498b-4f6c-4d20-8801-e37564ff8d7c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:09.717606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c498b-4f6c-4d20-8801-e37564ff8d7c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:09.718090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c498b-4f6c-4d20-8801-e37564ff8d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:09.739880 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4a5c498b-4f6c-4d20-8801-e37564ff8d7c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:09.741679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4a5c498b-4f6c-4d20-8801-e37564ff8d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:09.748063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:09.779064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:09.831700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 198/405] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:09.858604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c498b-4f6c-4d20-8801-e37564ff8d7c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:09.886429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea778c2-d3a7-4c22-a442-b1e9e6e2d014 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:09.970358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9147a7c-6caa-460a-b712-69a3d7a51f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:09.971151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9147a7c-6caa-460a-b712-69a3d7a51f0d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:09.971597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9147a7c-6caa-460a-b712-69a3d7a51f0d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:09.972034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9147a7c-6caa-460a-b712-69a3d7a51f0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:09.980313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c498b-4f6c-4d20-8801-e37564ff8d7c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:09.985140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9147a7c-6caa-460a-b712-69a3d7a51f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-a9147a7c-6caa-460a-b712-69a3d7a51f0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:09.993949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:09.994998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:09.995904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:10.003551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 208/406] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:10.004689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9147a7c-6caa-460a-b712-69a3d7a51f0d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.020545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905619a-462b-4c62-8e79-55da127ae2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.021360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905619a-462b-4c62-8e79-55da127ae2f4 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.021645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905619a-462b-4c62-8e79-55da127ae2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.021921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905619a-462b-4c62-8e79-55da127ae2f4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.036613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905619a-462b-4c62-8e79-55da127ae2f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-8905619a-462b-4c62-8e79-55da127ae2f4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:10.048507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:10.049452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:10.050499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 199/407] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:10.057013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905619a-462b-4c62-8e79-55da127ae2f4 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.096647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.142280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9147a7c-6caa-460a-b712-69a3d7a51f0d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.173315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.192302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905619a-462b-4c62-8e79-55da127ae2f4 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.196696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.198210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.198651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.199087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.199977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:10.200477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.202007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.217660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 209/408] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:10.219124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.262089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.264277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.264763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.266003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.267477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:10.267945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.269518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.288985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 200/409] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:10.294012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.381039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290dd4f-55a3-4c8b-ad71-9ac10cdc43dc None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.384662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.401561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e716c19-8900-473c-be32-c98b29281fce None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.435841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525c4d1-37b3-45a8-8e2b-f6170263672b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.437664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525c4d1-37b3-45a8-8e2b-f6170263672b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.437951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525c4d1-37b3-45a8-8e2b-f6170263672b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.438331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525c4d1-37b3-45a8-8e2b-f6170263672b None admin] PATH_INFO: `/v3/groups/9b471f0a21a04e4ba8e39be6f49c1810` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.439603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525c4d1-37b3-45a8-8e2b-f6170263672b None admin] RBAC: Authorizing `identity:get_group(group_id=9b471f0a21a04e4ba8e39be6f49c1810)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.441255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e525c4d1-37b3-45a8-8e2b-f6170263672b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.441591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a33ac-338d-40e6-9f73-0a22175ab3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.442456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a33ac-338d-40e6-9f73-0a22175ab3fc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.442740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a33ac-338d-40e6-9f73-0a22175ab3fc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.443014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a33ac-338d-40e6-9f73-0a22175ab3fc None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.444251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a33ac-338d-40e6-9f73-0a22175ab3fc None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.445762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a33ac-338d-40e6-9f73-0a22175ab3fc None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.447986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 210/410] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/groups/9b471f0a21a04e4ba8e39be6f49c1810 => generated 229 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:10.449556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308469b-1742-4f46-bf2e-644f6e891e38 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.449847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308469b-1742-4f46-bf2e-644f6e891e38 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.450123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308469b-1742-4f46-bf2e-644f6e891e38 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.451782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 211/411] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:10.452468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 201/412] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:10.452669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308469b-1742-4f46-bf2e-644f6e891e38 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.453338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a33ac-338d-40e6-9f73-0a22175ab3fc None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.511870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.588535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a33ac-338d-40e6-9f73-0a22175ab3fc None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.590685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308469b-1742-4f46-bf2e-644f6e891e38 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.625155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc0335-98e5-4a00-ae42-c7e3e253c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.625962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0335-98e5-4a00-ae42-c7e3e253c26d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.626254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0335-98e5-4a00-ae42-c7e3e253c26d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.626533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc0335-98e5-4a00-ae42-c7e3e253c26d None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.627737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc0335-98e5-4a00-ae42-c7e3e253c26d None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.629225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc0335-98e5-4a00-ae42-c7e3e253c26d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.635878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 202/413] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:10.637187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fbeb9-d438-4c2a-9f2b-235785e93ad7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.637484 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fbeb9-d438-4c2a-9f2b-235785e93ad7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.637766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631fbeb9-d438-4c2a-9f2b-235785e93ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.639829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c24f30-70ab-4398-b35d-844380b3d349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.640947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c24f30-70ab-4398-b35d-844380b3d349 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.641371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c24f30-70ab-4398-b35d-844380b3d349 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.641784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c24f30-70ab-4398-b35d-844380b3d349 None admin] PATH_INFO: `/v3/projects/1d6ac2b8c3194197aedc989787229928` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.644337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c24f30-70ab-4398-b35d-844380b3d349 None admin] RBAC: Authorizing `identity:get_project(project_id=1d6ac2b8c3194197aedc989787229928)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.645833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c24f30-70ab-4398-b35d-844380b3d349 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.647735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 212/414] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/projects/1d6ac2b8c3194197aedc989787229928 => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:10.648550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c24f30-70ab-4398-b35d-844380b3d349 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.662194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-631fbeb9-d438-4c2a-9f2b-235785e93ad7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:10.663976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-631fbeb9-d438-4c2a-9f2b-235785e93ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:10.685920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.768271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 203/415] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:10.769437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fbeb9-d438-4c2a-9f2b-235785e93ad7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.769963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c24f30-70ab-4398-b35d-844380b3d349 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:10.788994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:10.824975 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e5cd-8e5b-4602-b797-369933338468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:10.826234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e5cd-8e5b-4602-b797-369933338468 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:10.826716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e5cd-8e5b-4602-b797-369933338468 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:10.827206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536e5cd-8e5b-4602-b797-369933338468 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/groups/2f4fb549e19b4d239032ccc5654ba389/roles/20a552366bf14792b05decae218a79ee` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:10.830384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536e5cd-8e5b-4602-b797-369933338468 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a1c784d8b9e4949be326e17465c7aa7, group_id=2f4fb549e19b4d239032ccc5654ba389, role_id=20a552366bf14792b05decae218a79ee)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:10.832577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536e5cd-8e5b-4602-b797-369933338468 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:10.879473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 213/416] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 25 05:41:10 2022] PUT /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7/groups/2f4fb549e19b4d239032ccc5654ba389/roles/20a552366bf14792b05decae218a79ee => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:10.881226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e5cd-8e5b-4602-b797-369933338468 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:10.922779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631fbeb9-d438-4c2a-9f2b-235785e93ad7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.011627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536e5cd-8e5b-4602-b797-369933338468 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.029038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef86af-90a8-42cf-8643-83938f146cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.029826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef86af-90a8-42cf-8643-83938f146cc1 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.030115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef86af-90a8-42cf-8643-83938f146cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.030397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef86af-90a8-42cf-8643-83938f146cc1 None admin] PATH_INFO: `/v3/projects/1d6ac2b8c3194197aedc989787229928/groups/9b471f0a21a04e4ba8e39be6f49c1810/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.034626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef86af-90a8-42cf-8643-83938f146cc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d6ac2b8c3194197aedc989787229928, group_id=9b471f0a21a04e4ba8e39be6f49c1810, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.036178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef86af-90a8-42cf-8643-83938f146cc1 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.054028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3da281-1923-44aa-9b13-8bbc32d501c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.055305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3da281-1923-44aa-9b13-8bbc32d501c4 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.055845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3da281-1923-44aa-9b13-8bbc32d501c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.056328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3da281-1923-44aa-9b13-8bbc32d501c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.069443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3da281-1923-44aa-9b13-8bbc32d501c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-5f3da281-1923-44aa-9b13-8bbc32d501c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:11.078520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:11.079482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:11.080286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:11.082870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 214/417] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:11.090016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3da281-1923-44aa-9b13-8bbc32d501c4 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.128634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 204/418] 10.222.0.40 () {58 vars in 1442 bytes} [Mon Jul 25 05:41:10 2022] PUT /identity/v3/projects/1d6ac2b8c3194197aedc989787229928/groups/9b471f0a21a04e4ba8e39be6f49c1810/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:11.129676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.199546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a907644-34ab-460b-a6a5-ad3e3742b2f2 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.199736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a907644-34ab-460b-a6a5-ad3e3742b2f2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.200066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a907644-34ab-460b-a6a5-ad3e3742b2f2 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.201825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 205/419] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:11.209344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de090603-b42d-46fe-b9ac-e0f0b2c88b3b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.209634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de090603-b42d-46fe-b9ac-e0f0b2c88b3b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.210168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de090603-b42d-46fe-b9ac-e0f0b2c88b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.214078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3da281-1923-44aa-9b13-8bbc32d501c4 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.251007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-de090603-b42d-46fe-b9ac-e0f0b2c88b3b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:11.252526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-de090603-b42d-46fe-b9ac-e0f0b2c88b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:11.254725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.255840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.256579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.257044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.257463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.258942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] RBAC: Adding query filter params (name=member) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.259394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.261074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.272765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 215/420] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:11.279682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.370966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 206/421] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:11.387872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.402055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bee51-bf8c-4615-8111-3460b83c5845 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.414313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de090603-b42d-46fe-b9ac-e0f0b2c88b3b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.478089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e9cad-445d-4ea8-ab3b-85c73b46e724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.479604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e9cad-445d-4ea8-ab3b-85c73b46e724 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.479889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e9cad-445d-4ea8-ab3b-85c73b46e724 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.480186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e9cad-445d-4ea8-ab3b-85c73b46e724 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.568877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de090603-b42d-46fe-b9ac-e0f0b2c88b3b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.609624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11e418-0dfd-4b83-898f-f7e85080db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.610421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11e418-0dfd-4b83-898f-f7e85080db58 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.610758 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11e418-0dfd-4b83-898f-f7e85080db58 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.611035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11e418-0dfd-4b83-898f-f7e85080db58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.620520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.624667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11e418-0dfd-4b83-898f-f7e85080db58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-9e11e418-0dfd-4b83-898f-f7e85080db58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:11.633702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:11.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 25 05:41:11.635423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:11.636008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 207/422] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:11.642107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11e418-0dfd-4b83-898f-f7e85080db58 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.697498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e9cad-445d-4ea8-ab3b-85c73b46e724 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.698984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8e9cad-445d-4ea8-ab3b-85c73b46e724 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.754581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11e418-0dfd-4b83-898f-f7e85080db58 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:11.822806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:11.824681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:11.824681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:11.824681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:11.825646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:11.827180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:11.828802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:11.846240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 208/423] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:11.852446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.889888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:11.906369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 216/424] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:41:11.912599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e9cad-445d-4ea8-ab3b-85c73b46e724 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:11.994380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.019549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e9cad-445d-4ea8-ab3b-85c73b46e724 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.059159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec470e1-ac29-4ec7-b66b-80278bd8f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.060017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec470e1-ac29-4ec7-b66b-80278bd8f530 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.060474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec470e1-ac29-4ec7-b66b-80278bd8f530 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.060753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec470e1-ac29-4ec7-b66b-80278bd8f530 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.062195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec470e1-ac29-4ec7-b66b-80278bd8f530 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.063632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec470e1-ac29-4ec7-b66b-80278bd8f530 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.066296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 217/425] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.072666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec470e1-ac29-4ec7-b66b-80278bd8f530 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.115494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694fdbc-0a5f-4813-a957-bf816ae946a9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.169606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.179701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bc78b-54dd-4e4c-8854-389ebb58d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.180872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bc78b-54dd-4e4c-8854-389ebb58d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.181147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bc78b-54dd-4e4c-8854-389ebb58d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.181416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bc78b-54dd-4e4c-8854-389ebb58d4b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.228270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec470e1-ac29-4ec7-b66b-80278bd8f530 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.267363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bc78b-54dd-4e4c-8854-389ebb58d4b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.269662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bc78b-54dd-4e4c-8854-389ebb58d4b3 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.273641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a3411-6bb4-4cee-a992-24433fa4bbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.275018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a3411-6bb4-4cee-a992-24433fa4bbb3 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.275494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a3411-6bb4-4cee-a992-24433fa4bbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.275960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a3411-6bb4-4cee-a992-24433fa4bbb3 None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.277367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a3411-6bb4-4cee-a992-24433fa4bbb3 None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.279613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a3411-6bb4-4cee-a992-24433fa4bbb3 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.282389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 218/426] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.283949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec2665-6dd6-4744-932d-63f8c318a36e None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.284476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec2665-6dd6-4744-932d-63f8c318a36e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.284942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec2665-6dd6-4744-932d-63f8c318a36e None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.286702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 219/427] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:12.288016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec2665-6dd6-4744-932d-63f8c318a36e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.356225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.409048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 209/428] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:12.411064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291f6fe-8e1a-4a23-8621-b6bc3b412960 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.411361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291f6fe-8e1a-4a23-8621-b6bc3b412960 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.411645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291f6fe-8e1a-4a23-8621-b6bc3b412960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.433269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4291f6fe-8e1a-4a23-8621-b6bc3b412960 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:12.434476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4291f6fe-8e1a-4a23-8621-b6bc3b412960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:12.435692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec2665-6dd6-4744-932d-63f8c318a36e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.472906 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.482612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.485871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.485871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.485871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.485871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] RBAC: Adding query filter params (group.id=2f4fb549e19b4d239032ccc5654ba389, role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:12.485871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.486797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.500422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 220/429] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/role_assignments?group.id=2f4fb549e19b4d239032ccc5654ba389&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.501561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.512641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 210/430] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:12.545062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291f6fe-8e1a-4a23-8621-b6bc3b412960 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.664973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.673326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291f6fe-8e1a-4a23-8621-b6bc3b412960 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.709970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a865bc-e545-4de0-b751-8992ce13b513 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.717438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16788ea-af24-463c-8e32-8838d1c46480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.718271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16788ea-af24-463c-8e32-8838d1c46480 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.718556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16788ea-af24-463c-8e32-8838d1c46480 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.718834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16788ea-af24-463c-8e32-8838d1c46480 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.734541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16788ea-af24-463c-8e32-8838d1c46480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-e16788ea-af24-463c-8e32-8838d1c46480 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:12.745634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:12.746585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:12.747361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 211/431] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:12.754331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16788ea-af24-463c-8e32-8838d1c46480 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.762806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f324ec-8d55-4332-b8a6-58e991e30d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.763990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f324ec-8d55-4332-b8a6-58e991e30d33 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.764511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f324ec-8d55-4332-b8a6-58e991e30d33 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.764996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f324ec-8d55-4332-b8a6-58e991e30d33 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.766455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f324ec-8d55-4332-b8a6-58e991e30d33 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.768237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f324ec-8d55-4332-b8a6-58e991e30d33 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.770725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 221/432] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.781817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f324ec-8d55-4332-b8a6-58e991e30d33 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.860089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:12.886602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16788ea-af24-463c-8e32-8838d1c46480 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.919962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f324ec-8d55-4332-b8a6-58e991e30d33 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:12.931918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.932959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.933451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.933956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.935511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:12.936081 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.937759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.949536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 212/433] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.961807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.964474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4182ab-1a67-43c0-a070-6e50677c51e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:12.965857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4182ab-1a67-43c0-a070-6e50677c51e1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:12.966295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4182ab-1a67-43c0-a070-6e50677c51e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:12.966740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4182ab-1a67-43c0-a070-6e50677c51e1 None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:12.969415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4182ab-1a67-43c0-a070-6e50677c51e1 None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:12.971026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4182ab-1a67-43c0-a070-6e50677c51e1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:12.973620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 222/434] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:12.979401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4182ab-1a67-43c0-a070-6e50677c51e1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:12.988595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.094772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1e223-18b0-4395-8941-ba317635f906 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.116666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4182ab-1a67-43c0-a070-6e50677c51e1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.136836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91799c51-dcc7-416a-9c99-3f4711b4058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.137593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799c51-dcc7-416a-9c99-3f4711b4058e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.137992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799c51-dcc7-416a-9c99-3f4711b4058e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.138418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91799c51-dcc7-416a-9c99-3f4711b4058e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.152324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.170015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.170532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.170818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.171091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.171956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] RBAC: Adding query filter params (group.id=2f4fb549e19b4d239032ccc5654ba389, role.id=45ad0499b0b54c309464a2ebb8a3e5d0, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:13.174368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.175715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.192195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 223/435] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/role_assignments?group.id=2f4fb549e19b4d239032ccc5654ba389&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=45ad0499b0b54c309464a2ebb8a3e5d0 => generated 611 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:13.227656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91799c51-dcc7-416a-9c99-3f4711b4058e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.229585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91799c51-dcc7-416a-9c99-3f4711b4058e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.247335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.344816 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 213/436] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:13.350249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.371233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f0712-9ec8-4fa7-9952-256567e22266 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.371529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f0712-9ec8-4fa7-9952-256567e22266 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.371724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f0712-9ec8-4fa7-9952-256567e22266 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.373138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 214/437] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:13.378356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b3c57-636f-4226-9d92-8170692271f4 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.378640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b3c57-636f-4226-9d92-8170692271f4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.378922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b3c57-636f-4226-9d92-8170692271f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.403840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b11b3c57-636f-4226-9d92-8170692271f4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:13.410136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b11b3c57-636f-4226-9d92-8170692271f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:13.483660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e01eb-64ad-43a0-9a71-ac18e8df5f86 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.490940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 215/438] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:41:13.516629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b3c57-636f-4226-9d92-8170692271f4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.525226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acd9b3-753b-4fb1-888c-d8660dc00e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.526597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acd9b3-753b-4fb1-888c-d8660dc00e39 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.527050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acd9b3-753b-4fb1-888c-d8660dc00e39 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.527496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acd9b3-753b-4fb1-888c-d8660dc00e39 None admin] PATH_INFO: `/v3/projects/39a8b4919bb540a7a257d26089c8b3bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.529395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acd9b3-753b-4fb1-888c-d8660dc00e39 None admin] RBAC: Authorizing `identity:get_project(project_id=39a8b4919bb540a7a257d26089c8b3bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.532395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acd9b3-753b-4fb1-888c-d8660dc00e39 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.540490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 224/439] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/projects/39a8b4919bb540a7a257d26089c8b3bd => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:13.541666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acd9b3-753b-4fb1-888c-d8660dc00e39 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.574182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.601622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.627781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b3c57-636f-4226-9d92-8170692271f4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.641310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acd9b3-753b-4fb1-888c-d8660dc00e39 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.661830 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8fc9-bd8f-47fc-9cdf-65906bcd7820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.662644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8fc9-bd8f-47fc-9cdf-65906bcd7820 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.662932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8fc9-bd8f-47fc-9cdf-65906bcd7820 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.663209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8fc9-bd8f-47fc-9cdf-65906bcd7820 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.673227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed8fc9-bd8f-47fc-9cdf-65906bcd7820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:13.674736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c529dc-c3a2-40b4-be97-212619ee8ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.675580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c529dc-c3a2-40b4-be97-212619ee8ecd None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.675862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c529dc-c3a2-40b4-be97-212619ee8ecd None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.676167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c529dc-c3a2-40b4-be97-212619ee8ecd None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.677329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c529dc-c3a2-40b4-be97-212619ee8ecd None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.678778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c529dc-c3a2-40b4-be97-212619ee8ecd None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.680794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 225/440] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-a3ed8fc9-bd8f-47fc-9cdf-65906bcd7820 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:13.681539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:13.682462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:13.683713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:13.684760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12e4ba-090b-4866-8204-d675bdf14dd6 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.684760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12e4ba-090b-4866-8204-d675bdf14dd6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.684760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12e4ba-090b-4866-8204-d675bdf14dd6 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.685085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 216/441] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:13.685276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 226/442] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:13.686500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8fc9-bd8f-47fc-9cdf-65906bcd7820 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.689389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5264a-f078-4e8c-8a16-5b973f13d9f8 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.689854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5264a-f078-4e8c-8a16-5b973f13d9f8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.689957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a5264a-f078-4e8c-8a16-5b973f13d9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.706805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-e1a5264a-f078-4e8c-8a16-5b973f13d9f8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:13.708007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e1a5264a-f078-4e8c-8a16-5b973f13d9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:13.780342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 227/443] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:13.781190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5264a-f078-4e8c-8a16-5b973f13d9f8 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.791643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8fc9-bd8f-47fc-9cdf-65906bcd7820 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.795077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.829709 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.830367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.830653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.831028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.831868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] RBAC: Adding query filter params (group.id=2f4fb549e19b4d239032ccc5654ba389, role.id=20a552366bf14792b05decae218a79ee, scope.project.id=39a8b4919bb540a7a257d26089c8b3bd) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:13.832215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.833670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.840133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 217/444] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/role_assignments?group.id=2f4fb549e19b4d239032ccc5654ba389&scope.project.id=39a8b4919bb540a7a257d26089c8b3bd&role.id=20a552366bf14792b05decae218a79ee => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:13.841648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:13.845170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.888816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a5264a-f078-4e8c-8a16-5b973f13d9f8 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:13.926949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:13.927650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:13.927929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:13.928264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:13.929007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:13.929313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:13.931594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:13.941084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 228/445] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:13.947630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:13.966614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99385672-7e64-4554-bb0c-904ae01341ee None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.006912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e10c-f727-46b5-9cef-ef775957587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.008821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e10c-f727-46b5-9cef-ef775957587e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.009018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e10c-f727-46b5-9cef-ef775957587e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.009308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1e10c-f727-46b5-9cef-ef775957587e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.022270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1e10c-f727-46b5-9cef-ef775957587e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-5ab1e10c-f727-46b5-9cef-ef775957587e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:14.030227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:14.031181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:14.032180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:14.036194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 218/446] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:14.037524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e10c-f727-46b5-9cef-ef775957587e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.062701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb592a82-9ab5-4bd8-baf4-4d98e59569c8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.079751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.086205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.098990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad704a-c4a1-4dcf-a3c8-be1dc20c912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.099669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad704a-c4a1-4dcf-a3c8-be1dc20c912a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.099960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad704a-c4a1-4dcf-a3c8-be1dc20c912a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.100339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad704a-c4a1-4dcf-a3c8-be1dc20c912a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.169674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1e10c-f727-46b5-9cef-ef775957587e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.180620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad704a-c4a1-4dcf-a3c8-be1dc20c912a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.182380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad704a-c4a1-4dcf-a3c8-be1dc20c912a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.210222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.213026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.213026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.213026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.213026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:14.213026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.213994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.223472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 219/447] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:14.229174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.303016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 229/448] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:41:14.309513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad704a-c4a1-4dcf-a3c8-be1dc20c912a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.333433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.334307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.347379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e866616-c559-481e-b54f-d2f50a0d5da6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.383218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271c167-caaa-4d5f-9c01-d024b51e9607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.383872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271c167-caaa-4d5f-9c01-d024b51e9607 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.384193 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271c167-caaa-4d5f-9c01-d024b51e9607 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.384448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271c167-caaa-4d5f-9c01-d024b51e9607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.415568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad704a-c4a1-4dcf-a3c8-be1dc20c912a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.448882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c50dd0-ff0e-4dfc-9539-bcc09a47df97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.449671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c50dd0-ff0e-4dfc-9539-bcc09a47df97 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.449957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c50dd0-ff0e-4dfc-9539-bcc09a47df97 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.450238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c50dd0-ff0e-4dfc-9539-bcc09a47df97 None admin] PATH_INFO: `/v3/projects/2a1c784d8b9e4949be326e17465c7aa7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.450828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271c167-caaa-4d5f-9c01-d024b51e9607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.451429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c50dd0-ff0e-4dfc-9539-bcc09a47df97 None admin] RBAC: Authorizing `identity:get_project(project_id=2a1c784d8b9e4949be326e17465c7aa7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.452331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271c167-caaa-4d5f-9c01-d024b51e9607 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.452891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c50dd0-ff0e-4dfc-9539-bcc09a47df97 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.454766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 230/449] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/projects/2a1c784d8b9e4949be326e17465c7aa7 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:14.460068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c50dd0-ff0e-4dfc-9539-bcc09a47df97 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.539955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.546865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.550253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 220/450] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:41:14.555714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271c167-caaa-4d5f-9c01-d024b51e9607 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.559091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c50dd0-ff0e-4dfc-9539-bcc09a47df97 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.594024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf42019-fa46-4797-8cb3-768ef0c0a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.594875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf42019-fa46-4797-8cb3-768ef0c0a699 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.595154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf42019-fa46-4797-8cb3-768ef0c0a699 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.595426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf42019-fa46-4797-8cb3-768ef0c0a699 None admin] PATH_INFO: `/v3/groups/2f4fb549e19b4d239032ccc5654ba389` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.596622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf42019-fa46-4797-8cb3-768ef0c0a699 None admin] RBAC: Authorizing `identity:get_group(group_id=2f4fb549e19b4d239032ccc5654ba389)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.598056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf42019-fa46-4797-8cb3-768ef0c0a699 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.604322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 231/451] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/groups/2f4fb549e19b4d239032ccc5654ba389 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:14.605538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf42019-fa46-4797-8cb3-768ef0c0a699 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.657227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271c167-caaa-4d5f-9c01-d024b51e9607 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.691402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e947f9-f45a-4297-be46-dcd47750ef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.692330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e947f9-f45a-4297-be46-dcd47750ef1d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.692616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e947f9-f45a-4297-be46-dcd47750ef1d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.692913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e947f9-f45a-4297-be46-dcd47750ef1d None admin] PATH_INFO: `/v3/projects/1d6ac2b8c3194197aedc989787229928` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.694090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e947f9-f45a-4297-be46-dcd47750ef1d None admin] RBAC: Authorizing `identity:get_project(project_id=1d6ac2b8c3194197aedc989787229928)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.696065 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e947f9-f45a-4297-be46-dcd47750ef1d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.698092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 221/452] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/projects/1d6ac2b8c3194197aedc989787229928 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:14.703457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e947f9-f45a-4297-be46-dcd47750ef1d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.703650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf42019-fa46-4797-8cb3-768ef0c0a699 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.738605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.739248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.739529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.739800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.740669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] RBAC: Adding query filter params (group.id=2f4fb549e19b4d239032ccc5654ba389, role.id=20a552366bf14792b05decae218a79ee, scope.project.id=2a1c784d8b9e4949be326e17465c7aa7) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:14.740980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.742940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.748756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 232/453] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/role_assignments?group.id=2f4fb549e19b4d239032ccc5654ba389&scope.project.id=2a1c784d8b9e4949be326e17465c7aa7&role.id=20a552366bf14792b05decae218a79ee => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:14.767582 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.812224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e947f9-f45a-4297-be46-dcd47750ef1d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.848149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854970b-e484-4219-b67d-fdfc147d8bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.849026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854970b-e484-4219-b67d-fdfc147d8bdf None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.849310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854970b-e484-4219-b67d-fdfc147d8bdf None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:14.849588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854970b-e484-4219-b67d-fdfc147d8bdf None admin] PATH_INFO: `/v3/groups/9b471f0a21a04e4ba8e39be6f49c1810` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:14.850813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854970b-e484-4219-b67d-fdfc147d8bdf None admin] RBAC: Authorizing `identity:get_group(group_id=9b471f0a21a04e4ba8e39be6f49c1810)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:14.852402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854970b-e484-4219-b67d-fdfc147d8bdf None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:14.854091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 222/454] 10.222.0.40 () {56 vars in 1180 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/groups/9b471f0a21a04e4ba8e39be6f49c1810 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:14.860434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:14.883886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:14.965749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9587f-fe0d-448e-89a4-49ff95db5bf8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:14.999030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa12e1-3cf8-4818-af2f-1e49cb35ca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:14.999684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa12e1-3cf8-4818-af2f-1e49cb35ca49 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:14.999964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa12e1-3cf8-4818-af2f-1e49cb35ca49 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:15.000274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa12e1-3cf8-4818-af2f-1e49cb35ca49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:15.001115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa12e1-3cf8-4818-af2f-1e49cb35ca49 None admin] RBAC: Adding query filter params (group.id=9b471f0a21a04e4ba8e39be6f49c1810, role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=1d6ac2b8c3194197aedc989787229928) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:15.001425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa12e1-3cf8-4818-af2f-1e49cb35ca49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:15.002855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa12e1-3cf8-4818-af2f-1e49cb35ca49 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:15.009318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 233/455] 10.222.0.40 () {56 vars in 1376 bytes} [Mon Jul 25 05:41:14 2022] GET /identity/v3/role_assignments?group.id=9b471f0a21a04e4ba8e39be6f49c1810&scope.project.id=1d6ac2b8c3194197aedc989787229928&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:17.464887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03682bf5-7a49-4cb4-90a5-9eb41bb86fb5 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.465863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03682bf5-7a49-4cb4-90a5-9eb41bb86fb5 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.466378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03682bf5-7a49-4cb4-90a5-9eb41bb86fb5 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.468232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 223/456] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:17.473881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba251e86-6014-4089-ab9a-533df80ba67c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.474279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba251e86-6014-4089-ab9a-533df80ba67c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.474440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba251e86-6014-4089-ab9a-533df80ba67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.501355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ba251e86-6014-4089-ab9a-533df80ba67c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:17.502555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ba251e86-6014-4089-ab9a-533df80ba67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:17.594804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 234/457] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:17.646206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03682bf5-7a49-4cb4-90a5-9eb41bb86fb5 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.755633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2da77-a8cd-423b-af5c-2476b7804f29 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.759067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2da77-a8cd-423b-af5c-2476b7804f29 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.759067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2da77-a8cd-423b-af5c-2476b7804f29 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.759259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 235/458] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:17.770304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc374-3f8b-46b5-a3a5-cbe37ed82a9b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.770805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc374-3f8b-46b5-a3a5-cbe37ed82a9b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.771277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fc374-3f8b-46b5-a3a5-cbe37ed82a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.792502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ad5fc374-3f8b-46b5-a3a5-cbe37ed82a9b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:17.792833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03682bf5-7a49-4cb4-90a5-9eb41bb86fb5 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:17.794233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ad5fc374-3f8b-46b5-a3a5-cbe37ed82a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:17.812151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.836404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:17.847539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a884b2b-e0f1-41d1-a639-13fa4b5ec8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:17.848396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a884b2b-e0f1-41d1-a639-13fa4b5ec8d8 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.848683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a884b2b-e0f1-41d1-a639-13fa4b5ec8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.848959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a884b2b-e0f1-41d1-a639-13fa4b5ec8d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.860789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a884b2b-e0f1-41d1-a639-13fa4b5ec8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:17.862000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a884b2b-e0f1-41d1-a639-13fa4b5ec8d8 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-4a884b2b-e0f1-41d1-a639-13fa4b5ec8d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:17.881106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:17.882217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:17.883317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:17.883317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:17.883317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:17.883317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:17.883317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 224/459] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:17.884038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28f81-f9e2-4cac-a04b-ee7e903c7e71 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.884350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28f81-f9e2-4cac-a04b-ee7e903c7e71 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.884625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28f81-f9e2-4cac-a04b-ee7e903c7e71 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.890160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 225/460] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:17.890947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28f81-f9e2-4cac-a04b-ee7e903c7e71 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:17.894556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 236/461] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:17.896891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1f5c1-b2f2-420d-bf2b-be583c7b59f2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:17.897386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1f5c1-b2f2-420d-bf2b-be583c7b59f2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:17.897842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed1f5c1-b2f2-420d-bf2b-be583c7b59f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:17.920372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-8ed1f5c1-b2f2-420d-bf2b-be583c7b59f2 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:17.922444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8ed1f5c1-b2f2-420d-bf2b-be583c7b59f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:18.001437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28f81-f9e2-4cac-a04b-ee7e903c7e71 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.010775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 237/462] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:18.011847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1f5c1-b2f2-420d-bf2b-be583c7b59f2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.060830 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.061498 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.061799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.062322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.062882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:18.063187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.064602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.078545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 226/463] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:18.079226 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.105095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:18.131239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:18.135351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed1f5c1-b2f2-420d-bf2b-be583c7b59f2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.180309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.182127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.182621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.183073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.197332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.199812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.207798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6ee42-94b1-43f6-a662-11e8646c1197 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:18.222643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:18.223503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:18.224376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:18.224376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:18.224376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:18.224376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:18.225594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 238/464] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:18.226822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.252513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.253313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.253593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.253865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.267394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.269186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:18.293716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:18.294609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:18.295493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:18.295493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:18.295493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:18.295493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:18.303467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 227/465] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:18.303467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.355163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb8e63-4f2d-43cd-a413-bc1c9d52275b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.384410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:18.394930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a2a4c-085d-4756-848e-11f5cd1ed6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.395859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a2a4c-085d-4756-848e-11f5cd1ed6e7 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.399894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a2a4c-085d-4756-848e-11f5cd1ed6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.399894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a2a4c-085d-4756-848e-11f5cd1ed6e7 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.399894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a2a4c-085d-4756-848e-11f5cd1ed6e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.400182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:18.401073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a2a4c-085d-4756-848e-11f5cd1ed6e7 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.409892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7775e5-60a9-433c-9c6e-7720d4d296d7 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.445652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.446322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.446608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.447142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.447968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:18.448307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.449692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.461499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 228/466] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:18.462304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.518637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 239/467] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:18 2022] POST /identity/v3/users => generated 246 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:18.518637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f8658-e2df-457f-b15a-070ad0bc38a1 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.518637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f8658-e2df-457f-b15a-070ad0bc38a1 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.518637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344f8658-e2df-457f-b15a-070ad0bc38a1 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.518637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 240/468] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:18.518637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f8658-e2df-457f-b15a-070ad0bc38a1 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.603137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181cb3c-8701-41ba-994a-8fdef6af2885 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.644192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da92e0-87bb-4ce5-bcd8-b79055b993bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.644882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da92e0-87bb-4ce5-bcd8-b79055b993bf None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.645170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da92e0-87bb-4ce5-bcd8-b79055b993bf None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.645453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da92e0-87bb-4ce5-bcd8-b79055b993bf None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.646254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da92e0-87bb-4ce5-bcd8-b79055b993bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:18.646570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da92e0-87bb-4ce5-bcd8-b79055b993bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.648648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da92e0-87bb-4ce5-bcd8-b79055b993bf None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.661393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344f8658-e2df-457f-b15a-070ad0bc38a1 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.662659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 229/469] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:18.664707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971333c-5fd1-473f-b942-ab6ad48a7fb1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.665001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971333c-5fd1-473f-b942-ab6ad48a7fb1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.665342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971333c-5fd1-473f-b942-ab6ad48a7fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.685037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f971333c-5fd1-473f-b942-ab6ad48a7fb1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:18.686297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f971333c-5fd1-473f-b942-ab6ad48a7fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:18.689333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:18.712574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef8572-45be-4e8c-8596-3e507c27bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.713982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8572-45be-4e8c-8596-3e507c27bc30 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.713982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8572-45be-4e8c-8596-3e507c27bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.713982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef8572-45be-4e8c-8596-3e507c27bc30 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.714653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef8572-45be-4e8c-8596-3e507c27bc30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.716019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef8572-45be-4e8c-8596-3e507c27bc30 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.780403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 230/470] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:18.781184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971333c-5fd1-473f-b942-ab6ad48a7fb1 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.789101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 241/471] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:18 2022] POST /identity/v3/users => generated 248 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:18.811332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ded75-b33b-4fee-80cb-13409333e8cd None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.812081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ded75-b33b-4fee-80cb-13409333e8cd None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.812611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ded75-b33b-4fee-80cb-13409333e8cd None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.814550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 242/472] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:18.826305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171125a6-e2a7-4acb-877c-55c64238c77d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.832029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171125a6-e2a7-4acb-877c-55c64238c77d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.832029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171125a6-e2a7-4acb-877c-55c64238c77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.846866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-171125a6-e2a7-4acb-877c-55c64238c77d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:18.848483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-171125a6-e2a7-4acb-877c-55c64238c77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:18.891746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:18.918241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971333c-5fd1-473f-b942-ab6ad48a7fb1 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:18.942150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 243/473] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:41:18.943196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171125a6-e2a7-4acb-877c-55c64238c77d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:18.979903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c825283-3c6a-4071-9455-0a159c904ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:18.980637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c825283-3c6a-4071-9455-0a159c904ff8 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:18.980922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c825283-3c6a-4071-9455-0a159c904ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:18.981205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c825283-3c6a-4071-9455-0a159c904ff8 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:18.981927 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c825283-3c6a-4071-9455-0a159c904ff8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:18.983373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c825283-3c6a-4071-9455-0a159c904ff8 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:18.996321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:19.060573 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171125a6-e2a7-4acb-877c-55c64238c77d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:19.095000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 231/474] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:18 2022] POST /identity/v3/users => generated 248 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:19.095882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c825283-3c6a-4071-9455-0a159c904ff8 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:19.106980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:19.108290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.108794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.110240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.125193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:19.127186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:19.152616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:19.153538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:19.154382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:19.154382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:19.154382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:19.154382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:19.156608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 244/475] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:19.168228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:19.200343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:19.300048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636ce13-b57e-437c-9408-ac82ff50e874 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:19.323475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c825283-3c6a-4071-9455-0a159c904ff8 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:19.352635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:19.353711 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.355139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.355880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.358619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:19.359200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:19.363394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:19.379712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 245/476] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:19.385917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:19.391095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:19.392035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.392659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.393133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.394121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:19.395778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:19.432186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.sql.core [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:41:19.432186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 05:41:19.432186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [parameters: {'id': 'b046f715d20e4142a161cffeb1bbce89', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 05:41:19.432186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86791) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application result.read() Jul 25 05:41:19.457522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return ref.to_dict() Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:41:19.458459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application yield resource Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise self.value Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application session.commit() Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:41:19.459503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:41:19.460974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:41:19.462022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application result.read() Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:41:19.463014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [parameters: {'id': 'b046f715d20e4142a161cffeb1bbce89', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:19.463987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 25 05:41:19.468960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:19.469786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 232/477] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 25 05:41:19 2022] POST /identity/v3/roles => generated 144 bytes in 369 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:19.473318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:19.503521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:19.533972 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4120bb7-36a8-46af-a829-db4ef8e248cc None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:19.566029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:19.590883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acbdea-6664-4511-ac89-d8fb2e429e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:19.591593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acbdea-6664-4511-ac89-d8fb2e429e26 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.591946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acbdea-6664-4511-ac89-d8fb2e429e26 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.596337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acbdea-6664-4511-ac89-d8fb2e429e26 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.597114 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acbdea-6664-4511-ac89-d8fb2e429e26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:19.598595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acbdea-6664-4511-ac89-d8fb2e429e26 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:19.629425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf64e0-bc7d-459d-a618-1d852e59e7df None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:19.666322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300836c-90fe-42a1-adcb-e592dc068238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:19.667139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300836c-90fe-42a1-adcb-e592dc068238 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.667421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300836c-90fe-42a1-adcb-e592dc068238 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.667698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300836c-90fe-42a1-adcb-e592dc068238 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.678523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300836c-90fe-42a1-adcb-e592dc068238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:19.689403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 246/478] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:19 2022] POST /identity/v3/users => generated 249 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c300836c-90fe-42a1-adcb-e592dc068238 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:19.689939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:19.690811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:19.691673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 233/479] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:19.698112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acbdea-6664-4511-ac89-d8fb2e429e26 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:19.866409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acbdea-6664-4511-ac89-d8fb2e429e26 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:19.923310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f85d4d1-450f-4cbc-8770-a79a15f0090b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:19.927038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85d4d1-450f-4cbc-8770-a79a15f0090b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:19.927038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85d4d1-450f-4cbc-8770-a79a15f0090b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:19.927038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f85d4d1-450f-4cbc-8770-a79a15f0090b None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:19.927038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85d4d1-450f-4cbc-8770-a79a15f0090b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:19.927038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85d4d1-450f-4cbc-8770-a79a15f0090b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:19.932401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f85d4d1-450f-4cbc-8770-a79a15f0090b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:19.935284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:19.943089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 247/480] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:20.902170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27666b-0a74-4397-8271-2da267a1f6ef None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.903083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27666b-0a74-4397-8271-2da267a1f6ef None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.903537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27666b-0a74-4397-8271-2da267a1f6ef None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.905138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 234/481] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:20.910753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcc10-183c-439f-9a1e-8c2546562b3c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:20.911548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcc10-183c-439f-9a1e-8c2546562b3c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:20.912017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdcc10-183c-439f-9a1e-8c2546562b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:20.937138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-66fdcc10-183c-439f-9a1e-8c2546562b3c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:20.938655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-66fdcc10-183c-439f-9a1e-8c2546562b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:21.027748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 248/482] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:21.065323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27666b-0a74-4397-8271-2da267a1f6ef None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.197942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27666b-0a74-4397-8271-2da267a1f6ef None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.223600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.223790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2fc0c-b1a7-492b-bd8a-f844ca3a5d07 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.224053 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2fc0c-b1a7-492b-bd8a-f844ca3a5d07 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.224359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b2fc0c-b1a7-492b-bd8a-f844ca3a5d07 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.225877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 249/483] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:21.232730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f77b-6ec4-4236-b1a7-e47ace105d51 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.233017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f77b-6ec4-4236-b1a7-e47ace105d51 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.233291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444f77b-6ec4-4236-b1a7-e47ace105d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.247820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e450eee-144f-424c-840b-3481e7a8b499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.248971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e450eee-144f-424c-840b-3481e7a8b499 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.249495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e450eee-144f-424c-840b-3481e7a8b499 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.249970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e450eee-144f-424c-840b-3481e7a8b499 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.260135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4444f77b-6ec4-4236-b1a7-e47ace105d51 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:21.260617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4444f77b-6ec4-4236-b1a7-e47ace105d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:21.276229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e450eee-144f-424c-840b-3481e7a8b499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-4e450eee-144f-424c-840b-3481e7a8b499 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:21.284041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:21.284961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:21.285744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:21.296990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 235/484] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:21.297874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e450eee-144f-424c-840b-3481e7a8b499 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.408020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e450eee-144f-424c-840b-3481e7a8b499 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.443528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.444243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.444528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.444868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.446182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:21.446492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.447466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 250/485] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:21.448157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.467473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 236/486] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:21.472874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444f77b-6ec4-4236-b1a7-e47ace105d51 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.484721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.543191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.609412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea2bff-445b-45ef-bb92-6d41ba756a6f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.631595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.646410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37f7d5-b4b3-444a-b846-7af6ec2fa962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.647208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37f7d5-b4b3-444a-b846-7af6ec2fa962 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.647492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37f7d5-b4b3-444a-b846-7af6ec2fa962 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.648323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37f7d5-b4b3-444a-b846-7af6ec2fa962 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.660042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc37f7d5-b4b3-444a-b846-7af6ec2fa962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-dc37f7d5-b4b3-444a-b846-7af6ec2fa962 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:21.669849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:21.670719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:21.671505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 237/487] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:21.677780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37f7d5-b4b3-444a-b846-7af6ec2fa962 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.743620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444f77b-6ec4-4236-b1a7-e47ace105d51 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.793594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37f7d5-b4b3-444a-b846-7af6ec2fa962 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.805565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:21.809356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718ea31-e89e-4538-8d37-b7483ad3eb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.810162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718ea31-e89e-4538-8d37-b7483ad3eb1e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.810444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718ea31-e89e-4538-8d37-b7483ad3eb1e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.810719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718ea31-e89e-4538-8d37-b7483ad3eb1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.830667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:21.831335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.831619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.831893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.832718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:21.833023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.834500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.836031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718ea31-e89e-4538-8d37-b7483ad3eb1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:21.837509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718ea31-e89e-4538-8d37-b7483ad3eb1e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:21.848543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 238/488] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:21.849869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-f718ea31-e89e-4538-8d37-b7483ad3eb1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:21.878334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:21.880448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:21.882547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:21.882547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:21.882547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:21.882547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:21.882547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 251/489] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:21.882547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88fbdb-62c3-4fa3-b03b-f74258fec658 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:21.882547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88fbdb-62c3-4fa3-b03b-f74258fec658 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:21.882547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88fbdb-62c3-4fa3-b03b-f74258fec658 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:21.887710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 252/490] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:21.888698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88fbdb-62c3-4fa3-b03b-f74258fec658 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:21.962800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36501af4-5d58-44cc-8546-36933d8c9a14 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:21.994401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88fbdb-62c3-4fa3-b03b-f74258fec658 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.001213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75690f0-538c-4db3-a3fe-d2c8ba686ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.002012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75690f0-538c-4db3-a3fe-d2c8ba686ade None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.002293 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75690f0-538c-4db3-a3fe-d2c8ba686ade None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.002568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75690f0-538c-4db3-a3fe-d2c8ba686ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.008956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.015554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75690f0-538c-4db3-a3fe-d2c8ba686ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.017387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75690f0-538c-4db3-a3fe-d2c8ba686ade None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.033326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.033996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.034276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.034552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.035281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.035586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.036988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c75690f0-538c-4db3-a3fe-d2c8ba686ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:22.038004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:22.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:22.040700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:22.040700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:22.040700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:22.040700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:22.046985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 239/491] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:22.048669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f1a6-e206-4d8c-891c-ba71f992e795 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.049056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 253/492] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:22.049463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f1a6-e206-4d8c-891c-ba71f992e795 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.049945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.050056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f1a6-e206-4d8c-891c-ba71f992e795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.071479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2c63f1a6-e206-4d8c-891c-ba71f992e795 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:22.073234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2c63f1a6-e206-4d8c-891c-ba71f992e795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:22.086061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.150513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 240/493] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:22.151750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f1a6-e206-4d8c-891c-ba71f992e795 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.183367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7d4ab-0b59-4fba-a3f9-15a5cd1f2a88 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.221682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.222452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.222741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.223017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.223851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.224204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.225601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.240654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 254/494] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:22.241489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.278651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.282292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f1a6-e206-4d8c-891c-ba71f992e795 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.319507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.320826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.321322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.322051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.329842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.331529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:22.348711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:22.349672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:22.350575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:22.353647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 241/495] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:22.354371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43e557-0187-4333-bcbf-a78d38f57899 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.354998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.364365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.393197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1b348-05f2-41bd-aafc-b6f1f421036f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.393436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1b348-05f2-41bd-aafc-b6f1f421036f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.393730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1b348-05f2-41bd-aafc-b6f1f421036f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.394117 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1b348-05f2-41bd-aafc-b6f1f421036f None admin] PATH_INFO: `/v3/roles/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.406082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1b348-05f2-41bd-aafc-b6f1f421036f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-1bc1b348-05f2-41bd-aafc-b6f1f421036f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:22.415116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:22.416049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:22.417165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:22.420848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 255/496] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:22.421776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1b348-05f2-41bd-aafc-b6f1f421036f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.470899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265e126-c47a-4148-b312-e41187c2ce9e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.508594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.509403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.509681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.509962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.518364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.519827 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.521824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.531966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1b348-05f2-41bd-aafc-b6f1f421036f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:22.535055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:22.541479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 242/497] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:22.542732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.568071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601c7c0-60c8-4523-98b0-af171bf16386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.568766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601c7c0-60c8-4523-98b0-af171bf16386 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.569049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601c7c0-60c8-4523-98b0-af171bf16386 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.569324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601c7c0-60c8-4523-98b0-af171bf16386 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.570769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601c7c0-60c8-4523-98b0-af171bf16386 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.571016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601c7c0-60c8-4523-98b0-af171bf16386 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.572731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601c7c0-60c8-4523-98b0-af171bf16386 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.608089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 256/498] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:22.608638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.609406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aafddd-39a6-43c5-a8b5-e35f8704f716 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.609686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aafddd-39a6-43c5-a8b5-e35f8704f716 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.609960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aafddd-39a6-43c5-a8b5-e35f8704f716 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.611218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 257/499] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:22.611904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aafddd-39a6-43c5-a8b5-e35f8704f716 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.647519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d92c8d-c834-4ded-ba04-2c61ec68dbfc None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.680198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a491632-2bba-45eb-8071-97a584250a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.680861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a491632-2bba-45eb-8071-97a584250a28 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.681143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a491632-2bba-45eb-8071-97a584250a28 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.681416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a491632-2bba-45eb-8071-97a584250a28 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.682154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a491632-2bba-45eb-8071-97a584250a28 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.682457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a491632-2bba-45eb-8071-97a584250a28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.683833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a491632-2bba-45eb-8071-97a584250a28 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.691577 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 243/500] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:22.692804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd604-4d73-480b-b9b4-58904dae3a05 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.693090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd604-4d73-480b-b9b4-58904dae3a05 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.693368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcd604-4d73-480b-b9b4-58904dae3a05 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.695295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 244/501] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:22.695810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcd604-4d73-480b-b9b4-58904dae3a05 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.719616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aafddd-39a6-43c5-a8b5-e35f8704f716 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.753811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60337c-84fc-442b-a878-154d2da6b44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.754476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60337c-84fc-442b-a878-154d2da6b44d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.754755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60337c-84fc-442b-a878-154d2da6b44d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.755023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60337c-84fc-442b-a878-154d2da6b44d None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.755863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60337c-84fc-442b-a878-154d2da6b44d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:22.756196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60337c-84fc-442b-a878-154d2da6b44d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.758288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60337c-84fc-442b-a878-154d2da6b44d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.777862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 258/502] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:22.779031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a781d-c5f5-4197-b87c-a6959c817f32 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.779320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a781d-c5f5-4197-b87c-a6959c817f32 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.779705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a781d-c5f5-4197-b87c-a6959c817f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.790228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.796007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-5c7a781d-c5f5-4197-b87c-a6959c817f32 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:22.797249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5c7a781d-c5f5-4197-b87c-a6959c817f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:22.800903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcd604-4d73-480b-b9b4-58904dae3a05 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:22.828108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:22.836678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867874e8-5dd0-4de8-858a-f873bfcc5baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:22.837388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867874e8-5dd0-4de8-858a-f873bfcc5baf None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:22.837676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867874e8-5dd0-4de8-858a-f873bfcc5baf None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:22.840185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867874e8-5dd0-4de8-858a-f873bfcc5baf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:22.864936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 259/503] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:22.865732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a781d-c5f5-4197-b87c-a6959c817f32 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:22.905927 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867874e8-5dd0-4de8-858a-f873bfcc5baf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:22.907386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867874e8-5dd0-4de8-858a-f873bfcc5baf None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:22.969992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a781d-c5f5-4197-b87c-a6959c817f32 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.002735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 245/504] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:23.004004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e4313-93a7-4ccc-bd30-1fb24f9b3dbe None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.004317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e4313-93a7-4ccc-bd30-1fb24f9b3dbe None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.004594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e4313-93a7-4ccc-bd30-1fb24f9b3dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.009473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.010266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.010547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.010826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.013655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.021269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.022813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.022969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f82e4313-93a7-4ccc-bd30-1fb24f9b3dbe None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:23.024183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f82e4313-93a7-4ccc-bd30-1fb24f9b3dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:23.041328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:23.042205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:23.043112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:23.043112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:23.043112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:23.043112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:23.043112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 260/505] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:23.043407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.049950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.093626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 246/506] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:23.094379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e4313-93a7-4ccc-bd30-1fb24f9b3dbe None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.158499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed9e80-0e17-43e5-9334-9940819663f2 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.192211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e4313-93a7-4ccc-bd30-1fb24f9b3dbe None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.192680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82127d34-7b1b-4c79-8430-a2fb725b86af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.193332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82127d34-7b1b-4c79-8430-a2fb725b86af None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.193623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82127d34-7b1b-4c79-8430-a2fb725b86af None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.193898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82127d34-7b1b-4c79-8430-a2fb725b86af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.227501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.228337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.228628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.228910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.239058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.240554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.251651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:23.259948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:23.261022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:23.261889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:23.261889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:23.261889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:23.261889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:23.261889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 247/507] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:23.262285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.269472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82127d34-7b1b-4c79-8430-a2fb725b86af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.271018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82127d34-7b1b-4c79-8430-a2fb725b86af None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.274527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.362425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca7ff4-2b92-4671-8cad-a66e76e0f41f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.369389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 261/508] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:23.370369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82127d34-7b1b-4c79-8430-a2fb725b86af None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.395346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.396151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.396433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.396702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.407158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.408637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:23.425724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:23.426739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:23.427779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:23.427779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:23.427779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:23.427779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:23.427779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 248/509] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:23.427779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.459098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.471359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82127d34-7b1b-4c79-8430-a2fb725b86af None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.489232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.505690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.506352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.506638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.506908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.508291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:23.508603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.510128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.521455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 262/510] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:23.522249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.528134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1bb06-9290-4265-a4ef-6c15635a7c21 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.561634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbbcb9-1158-449f-8f9b-98872dcdd692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.562428 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbbcb9-1158-449f-8f9b-98872dcdd692 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.562707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbbcb9-1158-449f-8f9b-98872dcdd692 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.562983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dbbcb9-1158-449f-8f9b-98872dcdd692 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.573789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dbbcb9-1158-449f-8f9b-98872dcdd692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-e1dbbcb9-1158-449f-8f9b-98872dcdd692 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:23.581598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:23.582651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:23.583606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:23.587946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 249/511] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:23.588584 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbbcb9-1158-449f-8f9b-98872dcdd692 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.622735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ca51-f0cf-462d-92db-4b8eef631bbe None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.655145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.655807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.656092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.656406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.657140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:23.657447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.658836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.670906 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 263/512] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:23.671701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.681871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.692039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dbbcb9-1158-449f-8f9b-98872dcdd692 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.715136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.727345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.728162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.728457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.728739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.739171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.741007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:23.759147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:23.760256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:23.761309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:23.761309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:23.761309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:23.761309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:23.761309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 250/513] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:23.761654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.775853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573701e-9c15-4104-bc14-9edfd0e4b419 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.811766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.812453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.812735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.813014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.818207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:23.818207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.818207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.827614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 264/514] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:23.828434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.865799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6dfcc2-0df4-45dd-811a-d450c11dfdc8 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.894298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.899692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.900518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.900800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.901078 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.908624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.910062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:23.924942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:23.926037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 251/515] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:23.926950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:23.933582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5afa5-52ae-4417-a41f-ca76eeee9725 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:23.964811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:23.969700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:23.970451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:23.970703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:23.970977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:23.971727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] RBAC: Adding query filter params (name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:23.972030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:23.973432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:23.982711 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 265/516] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:23.983498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.031828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8243c49-186f-416a-9f75-3582753c52d9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.065872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2c1a-fb5e-4d7a-8407-e33ece514a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.066544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2c1a-fb5e-4d7a-8407-e33ece514a00 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.066831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2c1a-fb5e-4d7a-8407-e33ece514a00 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.067108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2c1a-fb5e-4d7a-8407-e33ece514a00 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.067805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d2c1a-fb5e-4d7a-8407-e33ece514a00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.069295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d2c1a-fb5e-4d7a-8407-e33ece514a00 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.081125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da4b961-f544-4fd5-b1fa-74cecf0abb2a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.109718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.115294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.115943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.116263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.117003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.121313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:24.121313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.121313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.130735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 266/517] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:24.131506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.148698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 252/518] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:23 2022] POST /identity/v3/users => generated 247 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:24.148698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2c1a-fb5e-4d7a-8407-e33ece514a00 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.252636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.259871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d28bb2-cc54-44e0-bce0-5fb65a025b9b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.265487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2c1a-fb5e-4d7a-8407-e33ece514a00 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.299544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.299891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.300203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.300489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.307532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.308204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.308315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.308505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.308777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.309611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:24.309727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.310037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.311407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:24.326522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:24.327415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:24.328312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 267/519] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:24.328949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.354172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 253/520] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:24.355025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.395041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.450178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20bedb-3c9c-4e0f-b593-5368dae8562c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.460783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc56d4-c223-4ad8-9116-4f8a7118cb2f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.485114 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.486188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.486667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.486950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.496858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.497658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.497942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.498217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.498621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.500087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.505743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.508180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:24.517926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:24.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:24.519958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:24.519958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:24.519958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:24.519958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:24.519958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 268/521] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:24.519958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:24.527898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:24.528954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:24.530019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:24.530019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:24.530019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:24.530019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:24.530019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:24.530019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:24.530019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:24.530019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:24.531819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 254/522] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:24.533284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.546126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.630053 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cc8ce-b89a-4ff8-acbd-ce6dc614b929 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.631776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.644415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d92ef-f0c5-4a18-9b48-1b0277347958 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.666546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.667226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.667518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.667797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.668623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:24.668939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.670440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.678538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d82ece-6d97-49fe-879c-84170b6ecfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.679731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d82ece-6d97-49fe-879c-84170b6ecfc6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.680010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d82ece-6d97-49fe-879c-84170b6ecfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.680312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d82ece-6d97-49fe-879c-84170b6ecfc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.693862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 269/523] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:24.694591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.754175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d82ece-6d97-49fe-879c-84170b6ecfc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.755574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d82ece-6d97-49fe-879c-84170b6ecfc6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.769879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.807279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fa131-33fd-40b7-b3e4-caf267a5ee31 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.843909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:24.844618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:24.844929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:24.845212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:24.845944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:24.846269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:24.847683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:24.860784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 270/524] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:24.862256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.879834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 255/525] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:24.879834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d82ece-6d97-49fe-879c-84170b6ecfc6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:24.891979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:24.977041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37babf-e045-47ad-a257-be6e68ec2f8b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.981786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d82ece-6d97-49fe-879c-84170b6ecfc6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:24.991461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.012647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.013410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.013709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.013988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.015030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.015490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.016915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.017790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.018445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.018723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.018997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.019813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=e6a0f84bf3f245bdab1391a2451cb4c0) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.020138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.021557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.027950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 256/526] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/role_assignments?user.id=e6a0f84bf3f245bdab1391a2451cb4c0&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:25.028760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.039542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 271/527] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:25.040763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.157255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.164126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a69c09-c3c8-4776-9eb9-3583f1f53b02 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.177795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2741e6-7929-4633-a92e-89c9d1fc34b8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.214487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.215699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.216289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.216892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.218809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.219695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.220049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.220458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.227578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.229889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.232575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.234175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:25.249054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:25.249981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:41:25.250870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:25.253393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:25.254220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:25.255062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:25.255062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:25.255062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:25.255062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:25.255789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 257/529] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:25.256794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.258725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 272/529] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:25.260826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.295714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.370274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9de39a-0190-42aa-9ca6-6e19bcfaaaa5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.377060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa773c-2dfa-452e-86d8-6d7abac182bb None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.407422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.408362 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.408649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.409089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.410063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=f8a47b60be1a45f9aabc262b2d04d596) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.410478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.412205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.415529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.416400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.416832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.417258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.418263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.418734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.419711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 258/530] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/role_assignments?user.id=f8a47b60be1a45f9aabc262b2d04d596&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:25.420514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.420725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.422776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.441899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 273/531] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:25.450256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.529725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.537031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f903a03-bbd2-4f17-a83a-baf096c390e7 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.608527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.609272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.609659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.611604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.612500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:25.612919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.615340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.643110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 259/532] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:25.646900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.655626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79d517-fd4a-4974-bd9e-e58a08e9f822 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.695744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4341f-ed30-4678-8ddb-8fd98cf27697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.696205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4341f-ed30-4678-8ddb-8fd98cf27697 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.698257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4341f-ed30-4678-8ddb-8fd98cf27697 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.698547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4341f-ed30-4678-8ddb-8fd98cf27697 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.798921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:25.809389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4341f-ed30-4678-8ddb-8fd98cf27697 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.815091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4341f-ed30-4678-8ddb-8fd98cf27697 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.883272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00b995-acb3-4f5b-929e-0bacf2f810d0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:25.949712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:25.950518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:25.950802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:25.951075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:25.963033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:25.964533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:25.984139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:25.985014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:25.985832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:25.985832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:25.985832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:25.985832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:25.985832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:25.985832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:25.985832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:25.985832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:25.987245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 260/533] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:25.988488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:25.992365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 274/534] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:25.997682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4341f-ed30-4678-8ddb-8fd98cf27697 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.007571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.097019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.107700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4341f-ed30-4678-8ddb-8fd98cf27697 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.159382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.161965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.162636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.164440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.179986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.182343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdd844-2e52-4503-902a-b5be83102ff0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.185683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:26.213608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:26.214505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:26.215406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:26.215406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:26.215406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:26.215406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:26.217508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 275/535] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:26.222895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.248396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.249402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.249901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.250391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.251492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:26.251998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.253750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.283590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 261/536] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:26.290374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.386680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b26e5-86b9-4733-8dd3-7b3aca1509e5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.414492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.426441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddcb96-bf0a-41c4-92de-5dea49730147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.427811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddcb96-bf0a-41c4-92de-5dea49730147 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.428326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddcb96-bf0a-41c4-92de-5dea49730147 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.438404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddcb96-bf0a-41c4-92de-5dea49730147 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.438404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddcb96-bf0a-41c4-92de-5dea49730147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:26.438404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddcb96-bf0a-41c4-92de-5dea49730147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.438404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddcb96-bf0a-41c4-92de-5dea49730147 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.440700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eca04d-6568-416a-89bb-6dd5cd2cdde4 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.448311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 276/537] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:26.449784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39509230-7c0f-4a8d-a82b-2d4ddf6726f3 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.450234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39509230-7c0f-4a8d-a82b-2d4ddf6726f3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.450672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39509230-7c0f-4a8d-a82b-2d4ddf6726f3 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.456452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 277/538] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:26.458699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39509230-7c0f-4a8d-a82b-2d4ddf6726f3 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.465494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.488808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc57b47-072c-4987-a20c-2ab8b621b2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.489763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57b47-072c-4987-a20c-2ab8b621b2cb None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.490271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57b47-072c-4987-a20c-2ab8b621b2cb None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.490751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc57b47-072c-4987-a20c-2ab8b621b2cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.491847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc57b47-072c-4987-a20c-2ab8b621b2cb None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=3c0883bc26a04de2916c78df342d2023) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:26.492400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc57b47-072c-4987-a20c-2ab8b621b2cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.494039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc57b47-072c-4987-a20c-2ab8b621b2cb None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.500593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 262/539] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/role_assignments?user.id=3c0883bc26a04de2916c78df342d2023&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:26.502007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8238c6-14df-4ca0-b54e-e281f4d2cce4 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.502513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8238c6-14df-4ca0-b54e-e281f4d2cce4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.502992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8238c6-14df-4ca0-b54e-e281f4d2cce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.521467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ce8238c6-14df-4ca0-b54e-e281f4d2cce4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:26.523695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ce8238c6-14df-4ca0-b54e-e281f4d2cce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:26.579389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39509230-7c0f-4a8d-a82b-2d4ddf6726f3 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.597290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 263/540] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:26.618784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.619613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.619897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.620264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.624808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8238c6-14df-4ca0-b54e-e281f4d2cce4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.629032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.630001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:26.645793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:26.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:26.648107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 278/541] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:26.653683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.705341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.721824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.741889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8238c6-14df-4ca0-b54e-e281f4d2cce4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.763471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8b0e0-13ad-428e-afe6-6832c20c4e36 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.781235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f991c3-c04b-4aa2-8da7-745aa2a9bf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.782315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f991c3-c04b-4aa2-8da7-745aa2a9bf06 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.782794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f991c3-c04b-4aa2-8da7-745aa2a9bf06 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.783276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f991c3-c04b-4aa2-8da7-745aa2a9bf06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.795588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f991c3-c04b-4aa2-8da7-745aa2a9bf06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:26.802721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.803386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.803666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.804508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.805441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:26.805768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-63f991c3-c04b-4aa2-8da7-745aa2a9bf06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:26.806514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:26.807345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:26.808579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:26.809203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.811084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 264/542] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:26.817063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f991c3-c04b-4aa2-8da7-745aa2a9bf06 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.828484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 279/543] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:26.829730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.927408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f991c3-c04b-4aa2-8da7-745aa2a9bf06 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.942506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eb055-a4f8-4b12-bac7-ee66b09c45eb None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:26.960636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.967228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.968161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.968624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.969079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.969361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:26.970219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:26.979005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.979005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.981983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:26.982716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:26.983000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:26.983279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:26.983662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 265/544] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:26.988184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=b1638c28af8240439d061a3ae0ddc38d) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:26.988184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:26.988184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:26.989466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:26.993012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 280/545] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/role_assignments?user.id=b1638c28af8240439d061a3ae0ddc38d&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:27.104230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684c779-3fd1-4ff7-a817-9a9a35edce63 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:27.141883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:27.143094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.143606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.144083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.156698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:27.158437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:27.177005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:27.177854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:27.179427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:27.179427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:27.179427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:27.179427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:27.179427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 266/546] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:27.184698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:27.306733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b6676-9b14-4a89-9e25-7d2fefcbdb25 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:27.343266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e5b59-eeb5-425b-bee9-9b1535b53a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:27.344312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e5b59-eeb5-425b-bee9-9b1535b53a6d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.344814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e5b59-eeb5-425b-bee9-9b1535b53a6d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.345284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e5b59-eeb5-425b-bee9-9b1535b53a6d None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.346228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e5b59-eeb5-425b-bee9-9b1535b53a6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:27.346746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e5b59-eeb5-425b-bee9-9b1535b53a6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:27.348393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e5b59-eeb5-425b-bee9-9b1535b53a6d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:27.361857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 281/547] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:27.367608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:27.391463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77053f8-630b-4af8-8072-9bff72af0a82 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.391790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77053f8-630b-4af8-8072-9bff72af0a82 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.392338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77053f8-630b-4af8-8072-9bff72af0a82 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.393809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 282/548] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:27.399825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84532-3fd1-494a-9edf-3e31d4ce8eee None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.400141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84532-3fd1-494a-9edf-3e31d4ce8eee None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.400446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f84532-3fd1-494a-9edf-3e31d4ce8eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.429559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:27.434901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-71f84532-3fd1-494a-9edf-3e31d4ce8eee None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:27.436144 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-71f84532-3fd1-494a-9edf-3e31d4ce8eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:27.525674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:27.529202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fd641-d6bf-4014-b1b6-c73087200b63 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:27.567108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 283/549] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:41:27.567701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:27.568532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.568819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.569097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] PATH_INFO: `/v3/users/swift` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.578492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:27.579970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:27.595391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:27.596376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:27.598326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 267/550] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:27.603384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f84532-3fd1-494a-9edf-3e31d4ce8eee None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:27.615212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:27.731798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f56908-7df7-46db-b29a-efe009b60a4d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:27.734318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:27.776165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c5c05-0d18-4903-b9d7-ef557cd74635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:27.776165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c5c05-0d18-4903-b9d7-ef557cd74635 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.776165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c5c05-0d18-4903-b9d7-ef557cd74635 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.776165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c5c05-0d18-4903-b9d7-ef557cd74635 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.785507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c5c05-0d18-4903-b9d7-ef557cd74635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-728c5c05-0d18-4903-b9d7-ef557cd74635 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:27.793919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:27.795004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:27.795802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:27.797124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 268/551] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:27.801679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c5c05-0d18-4903-b9d7-ef557cd74635 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:27.829137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f84532-3fd1-494a-9edf-3e31d4ce8eee None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:27.892283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:27.893099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:27.893385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:27.893667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:27.894675 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:27.894991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:27.896454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:27.916935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 284/552] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 25 05:41:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:27.922383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:27.961011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.006340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c5c05-0d18-4903-b9d7-ef557cd74635 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.032721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f821faa-6f09-46fc-8c09-83c1f3f763a9 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.073795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e52fc-9682-47dc-ac2c-ed3bbc2870f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.074821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e52fc-9682-47dc-ac2c-ed3bbc2870f8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.075251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e52fc-9682-47dc-ac2c-ed3bbc2870f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.075677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e52fc-9682-47dc-ac2c-ed3bbc2870f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.081674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.082347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.082637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.082918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.083679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:28.083988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.086324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.096551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 269/553] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:28.106541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.161847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.162188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e52fc-9682-47dc-ac2c-ed3bbc2870f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.164436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e52fc-9682-47dc-ac2c-ed3bbc2870f8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.207923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.240825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b080fed2-9668-4f53-9c7e-1bacab7d2d35 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.284871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.285902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.286371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.286851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.290694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 285/554] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:41:28.296184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df427-e92e-4c11-8386-c56689ebddf5 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.296184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df427-e92e-4c11-8386-c56689ebddf5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.296184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3df427-e92e-4c11-8386-c56689ebddf5 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.297901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 286/555] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:28.299082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19881dd3-4312-4982-b190-117259ff31be None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.299372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19881dd3-4312-4982-b190-117259ff31be None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.299746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19881dd3-4312-4982-b190-117259ff31be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.304889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.306627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.318011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-19881dd3-4312-4982-b190-117259ff31be None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:28.319258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-19881dd3-4312-4982-b190-117259ff31be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:28.326373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:28.327400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:28.328628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:28.328628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:28.328628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:28.328628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:28.335489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 270/556] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:28.336825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.397161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 287/557] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:28.399009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19881dd3-4312-4982-b190-117259ff31be None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.433248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.470545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd18a370-aa03-4bc5-b882-d1c1d2e23aac None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.488457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.518641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.519949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.520777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.521673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.526846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19881dd3-4312-4982-b190-117259ff31be None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.536424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.538799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:28.561380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:28.562784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:28.563662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:28.563662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:28.563662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:28.563662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:28.563889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.563889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.563889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.563889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.564493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:28.564991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.571774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 271/558] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:28.571774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.572799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.591504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 288/559] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:28.591504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.680191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a0a65a-1289-4ffc-a384-6f7bf5fd997d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.714410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.723596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f63eb8-74ba-4fd9-a130-2758b7f9b830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.725464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dac51-c0e5-4ad0-8289-6879172ca81c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.725656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f63eb8-74ba-4fd9-a130-2758b7f9b830 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.725656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f63eb8-74ba-4fd9-a130-2758b7f9b830 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.726385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f63eb8-74ba-4fd9-a130-2758b7f9b830 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.743254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f63eb8-74ba-4fd9-a130-2758b7f9b830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:28.751924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-29f63eb8-74ba-4fd9-a130-2758b7f9b830 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:28.755976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:28.757800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:28.758886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:28.766312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.766312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.766312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.766684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.766818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 272/560] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:28.767620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:28.768021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.768264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f63eb8-74ba-4fd9-a130-2758b7f9b830 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.769766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.792325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 289/561] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:28.793365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.901241 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df428035-420e-4e1f-8e3d-a48208a1f2e9 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.919157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f63eb8-74ba-4fd9-a130-2758b7f9b830 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:28.939832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.940686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.941066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.941441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.942324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] RBAC: Adding query filter params (name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:28.942725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.944273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:28.958699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 290/562] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:28.959696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:28.965482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:28.966495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:28.966929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:28.967352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:28.976946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:28.979467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:29.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:29.002204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:29.003352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:29.005642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 273/563] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:29.006608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.020315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.072331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.076680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a52708-1eb3-4858-9440-714dfe2fea8f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.115233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.117789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.119029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.120071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.126012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b424a-933d-4099-9a19-b956a37d2791 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.132441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.133043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:29.151037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:29.152051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 291/564] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:29.153499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.173586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb58e9-d819-4fab-b2b3-1aaf4edd5c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.174381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb58e9-d819-4fab-b2b3-1aaf4edd5c45 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.174663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb58e9-d819-4fab-b2b3-1aaf4edd5c45 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.174940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb58e9-d819-4fab-b2b3-1aaf4edd5c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.187290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb58e9-d819-4fab-b2b3-1aaf4edd5c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.188769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb58e9-d819-4fab-b2b3-1aaf4edd5c45 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-d0fb58e9-d819-4fab-b2b3-1aaf4edd5c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:29.208890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:29.210139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:29.211415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:29.211415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:29.211415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:29.211415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:29.211415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 274/565] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:29.213564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1adb-b3c7-4d4c-8a8d-a7ca45c755da None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.214008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1adb-b3c7-4d4c-8a8d-a7ca45c755da None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.214436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd1adb-b3c7-4d4c-8a8d-a7ca45c755da None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.216848 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 275/566] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:29.217930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd1adb-b3c7-4d4c-8a8d-a7ca45c755da None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.263587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00e659-a4d9-494e-9fad-4de439130ff9 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.273904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.299434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.300237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.300595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.300887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.301734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:29.302049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.303542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.323153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 292/567] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:29.323996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.339414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd1adb-b3c7-4d4c-8a8d-a7ca45c755da None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.341527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.374753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d546b85-988b-4013-9000-9a846eca8f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.375436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546b85-988b-4013-9000-9a846eca8f6e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.376232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546b85-988b-4013-9000-9a846eca8f6e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.376515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546b85-988b-4013-9000-9a846eca8f6e None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.377303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d546b85-988b-4013-9000-9a846eca8f6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:29.377618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d546b85-988b-4013-9000-9a846eca8f6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.379041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d546b85-988b-4013-9000-9a846eca8f6e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.390845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 276/568] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:29.392038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09dfea-fb85-46a2-afd2-a7f3c555ca45 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.392358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09dfea-fb85-46a2-afd2-a7f3c555ca45 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.392647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09dfea-fb85-46a2-afd2-a7f3c555ca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.408840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9a09dfea-fb85-46a2-afd2-a7f3c555ca45 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:29.409977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9a09dfea-fb85-46a2-afd2-a7f3c555ca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:29.431480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb650f6c-d31d-4788-b50d-8e27f7d9f69a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.465995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5415e3-377d-44da-8466-ecd2642ce364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.466657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5415e3-377d-44da-8466-ecd2642ce364 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.466942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5415e3-377d-44da-8466-ecd2642ce364 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.467221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5415e3-377d-44da-8466-ecd2642ce364 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.467950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5415e3-377d-44da-8466-ecd2642ce364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:29.468297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5415e3-377d-44da-8466-ecd2642ce364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.469690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5415e3-377d-44da-8466-ecd2642ce364 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.476802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 277/569] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:29.477710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09dfea-fb85-46a2-afd2-a7f3c555ca45 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.480886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 293/570] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:29.482179 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebea153-1c48-4b25-b82c-13ae476965dd None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.482477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebea153-1c48-4b25-b82c-13ae476965dd None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.482759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebea153-1c48-4b25-b82c-13ae476965dd None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.487862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 294/571] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:29.487862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebea153-1c48-4b25-b82c-13ae476965dd None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.512998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.580267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.587474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebea153-1c48-4b25-b82c-13ae476965dd None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.588908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09dfea-fb85-46a2-afd2-a7f3c555ca45 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.621331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.621980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.622086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb28ae-d171-451b-8499-32f4e58498ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.622262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.622536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.622763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb28ae-d171-451b-8499-32f4e58498ac None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.623050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb28ae-d171-451b-8499-32f4e58498ac None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.623328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb28ae-d171-451b-8499-32f4e58498ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.623428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=cf71e1ef5acb429b96219888830181d3) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:29.623690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.625227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.630711 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 295/572] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/role_assignments?user.id=cf71e1ef5acb429b96219888830181d3&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:29.631478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.634325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb28ae-d171-451b-8499-32f4e58498ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.635815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb28ae-d171-451b-8499-32f4e58498ac None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-1cfb28ae-d171-451b-8499-32f4e58498ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:29.655351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:29.656521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:29.657689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:29.657689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:29.657689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:29.657689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:29.657689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 278/573] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:29.658393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fbd1-8232-4cee-ae54-7c20398daad1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.658681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fbd1-8232-4cee-ae54-7c20398daad1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.659014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fbd1-8232-4cee-ae54-7c20398daad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.676321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9271fbd1-8232-4cee-ae54-7c20398daad1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:29.677570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9271fbd1-8232-4cee-ae54-7c20398daad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:29.745323 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.759521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be5c7e-de85-4875-9edc-921f92f2b3c1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.762086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 279/574] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:29.772225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271fbd1-8232-4cee-ae54-7c20398daad1 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.803342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.804730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.805287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.813399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:29.822438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:29.832075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:29.833229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:41:29.834391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:29.838387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 296/575] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:29.839253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.887296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:29.912690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271fbd1-8232-4cee-ae54-7c20398daad1 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:29.951603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867462d-808f-41bb-9a12-5f4d3619f978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:29.952436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867462d-808f-41bb-9a12-5f4d3619f978 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:29.952719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867462d-808f-41bb-9a12-5f4d3619f978 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:29.952989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867462d-808f-41bb-9a12-5f4d3619f978 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:29.963437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867462d-808f-41bb-9a12-5f4d3619f978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-f867462d-808f-41bb-9a12-5f4d3619f978 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:29.971153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:29.972087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:29.972963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 280/576] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:29.973672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867462d-808f-41bb-9a12-5f4d3619f978 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:29.996412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26664ec0-3493-4de9-8d91-a9c8a5d7e119 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.032790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.033621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.033915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.034254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.034988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.035534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.036992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.054849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 297/577] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.054849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.066277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.086967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867462d-808f-41bb-9a12-5f4d3619f978 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.125478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a87b9-4105-41f6-9e48-2e6d34d214fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.126276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a87b9-4105-41f6-9e48-2e6d34d214fb None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.126559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a87b9-4105-41f6-9e48-2e6d34d214fb None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.126833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a87b9-4105-41f6-9e48-2e6d34d214fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.128852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.138865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a87b9-4105-41f6-9e48-2e6d34d214fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-144a87b9-4105-41f6-9e48-2e6d34d214fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:30.146677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:30.148410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:30.149277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 281/578] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:30.150003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a87b9-4105-41f6-9e48-2e6d34d214fb None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.166964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddcb2d-eb46-42b7-b83e-91875a185268 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.204188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.204868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.205169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.205432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.206281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.206594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.208051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.226075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 298/579] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.226894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.257271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a87b9-4105-41f6-9e48-2e6d34d214fb None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.294164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.294683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.295414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.295708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.296044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.296901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.297211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.299515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.309279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 282/580] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.310122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.336889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14ea8d-8581-46b1-9146-188b7bcd80ea None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.372383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.373176 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.373452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.373729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.374175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.381775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.385102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:30.398445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:30.399341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:30.400347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 299/581] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:30.401450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.424973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4852-b6e4-4ac2-a4ab-72f3e71afdb3 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.461570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.462489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.462933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.463212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.463960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.464314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.465698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.478763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 283/582] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.478763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.511963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8246f971-9fe6-4637-b6ea-161b56a7e42e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.540729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.548915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.549709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.550008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.550260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.561475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.562992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.579250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ecf0-8dc6-4439-b19e-0454c4e60609 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:30.580282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:30.581165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:30.582082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:30.582082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:30.582082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:30.582082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:30.582082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 300/583] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:30.582399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.603610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.621356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.622157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.622443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.622721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.635293 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.636858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:30.655492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:30.656454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:30.661996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:30.661996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:30.661996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:30.661996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:30.661996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 284/584] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:30.661996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.702615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26769ed2-5992-4f1e-b8de-4a2bc93fd711 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.741889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.742695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.742981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.743261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.744425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.745034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.746679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.768338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 301/585] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.768338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.779079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d0559-8379-4ce9-8017-23e2bc7a4e41 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.814100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.817653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.818449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.818730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.819005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.831261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.832745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:30.849977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:30.851510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:30.852553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:30.852553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:30.852553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:30.852553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:30.852553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 285/586] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:30.852981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.882590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:30.883943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f0fd8-d8ac-42da-932e-b873317c8f62 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.921011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.921683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.921973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:30.922258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:30.922993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:30.923297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:30.924712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:30.941051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 302/587] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:30.941051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:30.964457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec4edc0-115c-468d-a141-f832ce4945e5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:30.999038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:30.999701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:30.999983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.000284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.000998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:31.001303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.002664 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.020270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 286/588] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:31.020270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.045574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:31.048524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8935a-3020-48a0-ba6c-97d50dda59bb None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.084727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfac8ca-0b1c-427d-bc18-dfdd32dee2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.085665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac8ca-0b1c-427d-bc18-dfdd32dee2ee None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.085956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac8ca-0b1c-427d-bc18-dfdd32dee2ee None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.086238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac8ca-0b1c-427d-bc18-dfdd32dee2ee None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/e6a0f84bf3f245bdab1391a2451cb4c0/roles/2c3d129fa9da4379acd24c88e17b7701` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.102027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfac8ca-0b1c-427d-bc18-dfdd32dee2ee None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=e6a0f84bf3f245bdab1391a2451cb4c0, role_id=2c3d129fa9da4379acd24c88e17b7701)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.103485 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfac8ca-0b1c-427d-bc18-dfdd32dee2ee None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.110200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:31.128440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0ffce-9682-4a2c-a26d-f1e98880426e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.160654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 303/589] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:30 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/e6a0f84bf3f245bdab1391a2451cb4c0/roles/2c3d129fa9da4379acd24c88e17b7701 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:31.162232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfac8ca-0b1c-427d-bc18-dfdd32dee2ee None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.176894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.177680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.178017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.178297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.179181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:31.179499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.181053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.213828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 287/590] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:31.213828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.358218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfac8ca-0b1c-427d-bc18-dfdd32dee2ee None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.424433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa0afc8-49b3-4424-93e3-7ccc2aef5169 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.437586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.438414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.438700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.438978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.455464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.461100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.469169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.470288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.470787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.471324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] PATH_INFO: `/v3/users/glance` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.480294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.482411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:31.487464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:31.488616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:31.489488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 304/591] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:31.489966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:31.512749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:31.513776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:41:31.514707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:31.519411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 288/592] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:31.520328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.523495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:31.527839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:31.692319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097599b-a86f-48be-a575-8bf35111cfac None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.701244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10804464-0f80-4c92-94c7-7bb4fccb92fa None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.732777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.733716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.734004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.734328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.735064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:31.735373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.736833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.737173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b4897-d4a5-4e9d-886c-018ca751418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.737970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b4897-d4a5-4e9d-886c-018ca751418c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.738261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b4897-d4a5-4e9d-886c-018ca751418c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.738536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b4897-d4a5-4e9d-886c-018ca751418c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.746042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b4897-d4a5-4e9d-886c-018ca751418c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.747490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b4897-d4a5-4e9d-886c-018ca751418c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.755650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 289/593] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:31.755650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-c37b4897-d4a5-4e9d-886c-018ca751418c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:31.765648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:31.766604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:31.767517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 305/594] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:31.768612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8c682-68d9-4d17-ba6f-3cace1142a89 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.768612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8c682-68d9-4d17-ba6f-3cace1142a89 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.768871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8c682-68d9-4d17-ba6f-3cace1142a89 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.773722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 306/595] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:31.773722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8c682-68d9-4d17-ba6f-3cace1142a89 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.829736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:31.845885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:31.902789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ac32f-5d77-47ba-b18c-90b073500971 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.906204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8c682-68d9-4d17-ba6f-3cace1142a89 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:31.938667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.939347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.939634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.939911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.940787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:31.941107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.942620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:31.947744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855b87e-c397-4dac-9376-64dd709340b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:31.949017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855b87e-c397-4dac-9376-64dd709340b8 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:31.949497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855b87e-c397-4dac-9376-64dd709340b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:31.950075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855b87e-c397-4dac-9376-64dd709340b8 None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/f8a47b60be1a45f9aabc262b2d04d596/roles/2c3d129fa9da4379acd24c88e17b7701` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:31.960818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 290/596] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:31.961732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:31.971074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855b87e-c397-4dac-9376-64dd709340b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=f8a47b60be1a45f9aabc262b2d04d596, role_id=2c3d129fa9da4379acd24c88e17b7701)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:31.972856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855b87e-c397-4dac-9376-64dd709340b8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.025916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 307/597] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:31 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/f8a47b60be1a45f9aabc262b2d04d596/roles/2c3d129fa9da4379acd24c88e17b7701 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:32.027640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c5aa0-7bee-4fcb-8e2e-af7cd42976b6 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.027931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c5aa0-7bee-4fcb-8e2e-af7cd42976b6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.028234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c5aa0-7bee-4fcb-8e2e-af7cd42976b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.054811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-e58c5aa0-7bee-4fcb-8e2e-af7cd42976b6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:32.056293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e58c5aa0-7bee-4fcb-8e2e-af7cd42976b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:32.077240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70143b-285a-44a7-9fd4-18350191a943 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.102439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.120447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.121357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.121808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.122183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.123147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:32.123754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.125393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.145699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 308/598] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:32.151255 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 291/599] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Jul 25 05:41:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:32.151967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c5aa0-7bee-4fcb-8e2e-af7cd42976b6 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.152533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.219349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.280838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c5aa0-7bee-4fcb-8e2e-af7cd42976b6 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.281325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be633aa-ffb2-450c-9c60-037d4dc75f11 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.318649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.320074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.320571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.320846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.329389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.330934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.331638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.332081 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.332385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.334040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.347193 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.349014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:32.355428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:32.356676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:32.357787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:32.357787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:32.357787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:32.357787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:32.357787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 309/600] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:32.357787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:32.374426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:32.375565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:32.376525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:32.376525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:32.376525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:32.376525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:32.377616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 292/601] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:32.382819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.424695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.479387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461d526-7bce-4f82-8635-3aa49dbbbd4d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.481452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.520663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbf2e2-69c7-4478-9505-f4a62b1b6a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.521469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbf2e2-69c7-4478-9505-f4a62b1b6a36 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.521757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbf2e2-69c7-4478-9505-f4a62b1b6a36 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.522033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbf2e2-69c7-4478-9505-f4a62b1b6a36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.524182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266010ea-3fa6-404a-8ee7-316a53f8fd4b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.540516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbf2e2-69c7-4478-9505-f4a62b1b6a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-f8fbf2e2-69c7-4478-9505-f4a62b1b6a36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:32.550895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:32.552021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:32.553124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:32.556237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 310/602] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:32.557349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbf2e2-69c7-4478-9505-f4a62b1b6a36 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.574177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.574999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.575391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.575777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.576767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:32.577183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.578737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.592734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 293/603] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:32.593592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.692749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.698591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbf2e2-69c7-4478-9505-f4a62b1b6a36 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.707251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffb702-ad7d-421b-adf8-ed3e5ac67321 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.738906 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.740229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.741762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.742543 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.743758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:32.744715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.746601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.749278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.750421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.750704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.750993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.751757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:32.752070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.753471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.768147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 294/604] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:32.768147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.771433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 311/605] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:32.772874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.798448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:32.879311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00241fa0-5389-4c74-9467-47c2215e2350 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.884356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca1e9f-2178-4fa9-9c1e-987de6144da6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:32.915907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.916780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.917067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.917344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.923663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:32.924838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:32.925322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:32.925828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:32.928331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.929842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.933885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:32.935613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:32.948431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:32.949842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:32.951174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:32.951174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:32.951174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:32.951174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:32.954366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 312/606] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:32.955209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:32.957311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:32.958165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:32.959052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:32.959052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:32.959052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:32.959052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:32.959052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:32.959052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:32.959052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:32.959052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:32.962795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 295/607] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:32.963874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:32.981189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.039024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.063618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3259f65-59bc-43c6-9ec8-27c13a2e9bff None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.077169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543315c-f316-4a74-a7be-a00f2e7ccede None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.099891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.100997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.101653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.101653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.108580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.109992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.117182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.118120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.118604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.119142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.120216 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:33.120730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.122295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:33.124473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:33.125357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:33.126233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 313/608] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:33.126710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.139680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 296/609] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:33.141004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.220162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.235596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f90ab1-242e-4e90-b6cb-e9672ffeab1a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.254505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c8b55-a613-402b-8dbb-19b6cfbb0517 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.273999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.274667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.274958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.275240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.276051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:33.276396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.278888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.280745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.292366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 314/610] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:33.292520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.299881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba469157-9bf0-4ace-85fb-beb25d18c666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.300803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba469157-9bf0-4ace-85fb-beb25d18c666 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.301235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba469157-9bf0-4ace-85fb-beb25d18c666 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.301658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba469157-9bf0-4ace-85fb-beb25d18c666 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.302595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba469157-9bf0-4ace-85fb-beb25d18c666 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:33.303051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba469157-9bf0-4ace-85fb-beb25d18c666 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.304743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba469157-9bf0-4ace-85fb-beb25d18c666 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.317585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 297/611] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:33.319649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1c1a-747b-4980-8482-cb3afae321ba None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.319952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1c1a-747b-4980-8482-cb3afae321ba None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.320267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be1c1a-747b-4980-8482-cb3afae321ba None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.324486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 298/612] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:33.324486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be1c1a-747b-4980-8482-cb3afae321ba None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.402729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82876d59-6925-4ce6-a682-72a64035ce93 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.429343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be1c1a-747b-4980-8482-cb3afae321ba None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.437754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.438540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.438824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.439094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.447319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.448831 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.460575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:33.464067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:33.465003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:33.465922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 315/613] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:33.467202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.471450 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97f474-aa3f-497a-b472-8ce9f76fe7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.474497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f474-aa3f-497a-b472-8ce9f76fe7d5 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.474497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f474-aa3f-497a-b472-8ce9f76fe7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.474497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97f474-aa3f-497a-b472-8ce9f76fe7d5 None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/3c0883bc26a04de2916c78df342d2023/roles/2c3d129fa9da4379acd24c88e17b7701` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.487691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97f474-aa3f-497a-b472-8ce9f76fe7d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=3c0883bc26a04de2916c78df342d2023, role_id=2c3d129fa9da4379acd24c88e17b7701)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.490218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97f474-aa3f-497a-b472-8ce9f76fe7d5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.523596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.568839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 299/614] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:33 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/3c0883bc26a04de2916c78df342d2023/roles/2c3d129fa9da4379acd24c88e17b7701 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:33.570869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564b9a9-8609-4737-b22d-b7f44cdf5985 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.571413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564b9a9-8609-4737-b22d-b7f44cdf5985 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.571705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564b9a9-8609-4737-b22d-b7f44cdf5985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.572756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24aa57-38f4-4b5e-87d2-e3a533a37a8a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.590024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4564b9a9-8609-4737-b22d-b7f44cdf5985 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:33.591364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4564b9a9-8609-4737-b22d-b7f44cdf5985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:33.632863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec295cb7-5532-483f-8f3b-7e8eb8528378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.633843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec295cb7-5532-483f-8f3b-7e8eb8528378 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.634132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec295cb7-5532-483f-8f3b-7e8eb8528378 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.634415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec295cb7-5532-483f-8f3b-7e8eb8528378 None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/b1638c28af8240439d061a3ae0ddc38d/roles/2c3d129fa9da4379acd24c88e17b7701` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.653066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec295cb7-5532-483f-8f3b-7e8eb8528378 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=b1638c28af8240439d061a3ae0ddc38d, role_id=2c3d129fa9da4379acd24c88e17b7701)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.654697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec295cb7-5532-483f-8f3b-7e8eb8528378 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.687702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 300/615] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:33.689131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564b9a9-8609-4737-b22d-b7f44cdf5985 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.713801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 316/616] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:33 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/b1638c28af8240439d061a3ae0ddc38d/roles/2c3d129fa9da4379acd24c88e17b7701 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:33.721921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec295cb7-5532-483f-8f3b-7e8eb8528378 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.813827 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564b9a9-8609-4737-b22d-b7f44cdf5985 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.829632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.853060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec295cb7-5532-483f-8f3b-7e8eb8528378 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:33.855024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.855718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.856002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.856149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:33.856314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.857421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:33.857735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:33.859210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:33.880845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 301/617] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:33.886799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:33.911104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566c9d3-d097-40e5-87e7-6563f5f275d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:33.911917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566c9d3-d097-40e5-87e7-6563f5f275d2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:33.912224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566c9d3-d097-40e5-87e7-6563f5f275d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:33.912507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566c9d3-d097-40e5-87e7-6563f5f275d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:33.929334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566c9d3-d097-40e5-87e7-6563f5f275d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-4566c9d3-d097-40e5-87e7-6563f5f275d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:33.939551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:33.940581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:33.941444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:33.942257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 317/618] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:33.949500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566c9d3-d097-40e5-87e7-6563f5f275d2 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.065468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1f6da-74e5-4658-9d20-85626f65cbdc None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.102501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.103177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.103465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.104014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.110021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566c9d3-d097-40e5-87e7-6563f5f275d2 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.120140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.120140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:34.136140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:34.137106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:34.138009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:34.138009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:34.138009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:34.138009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:34.138009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 302/619] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:34.142891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.187307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.187985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.188384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.188661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.189715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] RBAC: Adding query filter params (name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:34.190026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.191411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.203050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 318/620] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:34.206141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.211553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.269299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.294166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac98f9d-a770-42e6-93a6-e752a5fd9808 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.334108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b3a42-4cfe-4eeb-bd20-f488fc56356a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.336523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b3a42-4cfe-4eeb-bd20-f488fc56356a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.337327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b3a42-4cfe-4eeb-bd20-f488fc56356a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.337813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b3a42-4cfe-4eeb-bd20-f488fc56356a None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.339944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b3a42-4cfe-4eeb-bd20-f488fc56356a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:34.340490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b3a42-4cfe-4eeb-bd20-f488fc56356a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.342095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b3a42-4cfe-4eeb-bd20-f488fc56356a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.359729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 303/621] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:34.361396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005702-811d-4db6-8b48-80acf98d520c None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.362465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005702-811d-4db6-8b48-80acf98d520c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.362959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a005702-811d-4db6-8b48-80acf98d520c None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.365330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 304/622] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:34.367205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a005702-811d-4db6-8b48-80acf98d520c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.385584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6380651-d389-4252-a1dc-e196fd5820ba None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.429007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0000ab96-f50e-4bd8-aa14-54e00b5bea61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.430038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000ab96-f50e-4bd8-aa14-54e00b5bea61 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.430329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000ab96-f50e-4bd8-aa14-54e00b5bea61 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.430608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0000ab96-f50e-4bd8-aa14-54e00b5bea61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.443009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000ab96-f50e-4bd8-aa14-54e00b5bea61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.445754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0000ab96-f50e-4bd8-aa14-54e00b5bea61 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-0000ab96-f50e-4bd8-aa14-54e00b5bea61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:34.469611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:34.470566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:34.471459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:34.471459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:34.471459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:34.471459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:34.476255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 319/623] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:34.478638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164dede-b24a-4227-b218-2cfc9996ac58 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.478928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164dede-b24a-4227-b218-2cfc9996ac58 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.479206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164dede-b24a-4227-b218-2cfc9996ac58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.483083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.498027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d164dede-b24a-4227-b218-2cfc9996ac58 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:34.499559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d164dede-b24a-4227-b218-2cfc9996ac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:34.502869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a005702-811d-4db6-8b48-80acf98d520c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.539746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.540694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.541061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.541418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.549147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.550657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:34.565939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:34.566885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:34.567795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 305/624] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:34.568355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.583128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 320/625] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:34.584720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164dede-b24a-4227-b218-2cfc9996ac58 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.595629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.696986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164dede-b24a-4227-b218-2cfc9996ac58 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.717587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b399a-7d06-49fe-906c-52531eaf46ca None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.734565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.735501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.735993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.736537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.737596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:34.738108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.739774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.752994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 321/626] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:34.754126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.758143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.759653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.759946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.760274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.761188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:34.761507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.762879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:34.781365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 306/627] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:34.784553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.788970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.838123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:34.877508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074c6b2-4c19-4f44-aabd-9fc321cacd8f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.912296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a0c7e-a120-4c73-a669-596ad613fc29 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:34.919246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e8285-d452-4994-9779-dd474357ca70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.920757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e8285-d452-4994-9779-dd474357ca70 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.921243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e8285-d452-4994-9779-dd474357ca70 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.921729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e8285-d452-4994-9779-dd474357ca70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.936005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e8285-d452-4994-9779-dd474357ca70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-110e8285-d452-4994-9779-dd474357ca70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:34.947125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:34.948070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:34.949020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:34.949671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1920-198f-4b4e-9148-e98c77847aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:34.949671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1920-198f-4b4e-9148-e98c77847aad None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:34.949671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1920-198f-4b4e-9148-e98c77847aad None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:34.949671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1920-198f-4b4e-9148-e98c77847aad None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/cf71e1ef5acb429b96219888830181d3/roles/2c3d129fa9da4379acd24c88e17b7701` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:34.957339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 322/628] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:34.957512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e8285-d452-4994-9779-dd474357ca70 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:34.976155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b1920-198f-4b4e-9148-e98c77847aad None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=cf71e1ef5acb429b96219888830181d3, role_id=2c3d129fa9da4379acd24c88e17b7701)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:34.976155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b1920-198f-4b4e-9148-e98c77847aad None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.032221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 307/629] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:34 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/cf71e1ef5acb429b96219888830181d3/roles/2c3d129fa9da4379acd24c88e17b7701 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:35.033180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1920-198f-4b4e-9148-e98c77847aad None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.116764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e8285-d452-4994-9779-dd474357ca70 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.142546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.173335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.196897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.197770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.197997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.198276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.212425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.213919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.226482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1920-198f-4b4e-9148-e98c77847aad None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:35.232135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:35.233077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:35.234015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 323/630] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:35.237212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.280195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.280879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.281158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.282352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.283115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:35.283426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.284937 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.298236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 308/631] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:35.304045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.362868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9b783-e21f-4309-864c-95a869b52f51 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.405529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.406314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.406604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.406882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.407731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:35.408041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.409526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.431233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 324/632] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:35.436769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.444899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2849e-be6f-4dd0-b954-b2b4b0f7c24f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.465282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.490083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.494154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.494968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.495328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.495490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.511604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.514377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:35.541419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:35.543472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:35.544778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:35.544778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:35.544778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:35.544778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:35.544778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 309/633] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:35.551175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.555770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93c59e-280a-4058-9fdd-974bf58f8093 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.634704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ae34a-c349-4c6e-a2e0-eea26e27df12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.635489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ae34a-c349-4c6e-a2e0-eea26e27df12 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.635741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ae34a-c349-4c6e-a2e0-eea26e27df12 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.636026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ae34a-c349-4c6e-a2e0-eea26e27df12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.669905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c04bd8-b0e7-4f8e-a5fb-658ed44b8c56 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.707337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.708011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.708322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.708600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.709333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:35.709739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.712297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.724809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 310/634] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:35.731088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.789193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ae34a-c349-4c6e-a2e0-eea26e27df12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.790679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ae34a-c349-4c6e-a2e0-eea26e27df12 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.796494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.877348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:35.905460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6fcea7-0425-4333-8b2a-173c4ec193b8 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:35.947268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:35.948075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:35.948380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:35.948655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:35.963990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:35.965475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:35.987112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:35.988127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:35.989092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 311/635] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:35.993889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 325/636] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 25 05:41:35.995044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:35.998845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ae34a-c349-4c6e-a2e0-eea26e27df12 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.091002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.109092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f13bb-a7a7-417b-99e5-3a47f423500c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.132479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ae34a-c349-4c6e-a2e0-eea26e27df12 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.148160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9dab9-d763-4d71-baf0-bcb4b513d70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.148842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9dab9-d763-4d71-baf0-bcb4b513d70e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.149126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9dab9-d763-4d71-baf0-bcb4b513d70e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.149400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9dab9-d763-4d71-baf0-bcb4b513d70e None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.150237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9dab9-d763-4d71-baf0-bcb4b513d70e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:36.150543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9dab9-d763-4d71-baf0-bcb4b513d70e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.151967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9dab9-d763-4d71-baf0-bcb4b513d70e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.168799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e29a9-a6a2-48d7-bb65-9ac56e0ae4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.169602 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e29a9-a6a2-48d7-bb65-9ac56e0ae4d0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.169887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e29a9-a6a2-48d7-bb65-9ac56e0ae4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.170162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e29a9-a6a2-48d7-bb65-9ac56e0ae4d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.175147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 312/637] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:36.175147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8b064-1d64-43af-8611-c7626918de92 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.175147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8b064-1d64-43af-8611-c7626918de92 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.175147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a8b064-1d64-43af-8611-c7626918de92 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.175147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 313/638] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:36.175147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba2f9-3d8c-450b-afa6-e46a1f5cb37f None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.175581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba2f9-3d8c-450b-afa6-e46a1f5cb37f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.175581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beba2f9-3d8c-450b-afa6-e46a1f5cb37f None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.180248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 314/639] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:36.181884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beba2f9-3d8c-450b-afa6-e46a1f5cb37f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.188793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e29a9-a6a2-48d7-bb65-9ac56e0ae4d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.190268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e29a9-a6a2-48d7-bb65-9ac56e0ae4d0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.198657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-625e29a9-a6a2-48d7-bb65-9ac56e0ae4d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:36.211978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:36.218436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:36.219373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:36.219373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:36.219373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:36.219373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:36.219373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 326/640] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:36.219373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879366a-6223-4a1b-b70b-43bdf52ccac3 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.219373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879366a-6223-4a1b-b70b-43bdf52ccac3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.219373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879366a-6223-4a1b-b70b-43bdf52ccac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.234915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9879366a-6223-4a1b-b70b-43bdf52ccac3 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:36.236536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9879366a-6223-4a1b-b70b-43bdf52ccac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:36.299664 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beba2f9-3d8c-450b-afa6-e46a1f5cb37f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.316469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 327/641] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:36.316469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1330e-35d5-4558-8c01-54959e005a4e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.316469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1330e-35d5-4558-8c01-54959e005a4e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.316469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c1330e-35d5-4558-8c01-54959e005a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.336672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-98c1330e-35d5-4558-8c01-54959e005a4e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:36.338184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-98c1330e-35d5-4558-8c01-54959e005a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:36.342524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce7474-0fc2-4984-82c6-7bce40f37ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.343347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce7474-0fc2-4984-82c6-7bce40f37ac1 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.343803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce7474-0fc2-4984-82c6-7bce40f37ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.344305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce7474-0fc2-4984-82c6-7bce40f37ac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.359969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.416266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 328/642] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:36.417146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1330e-35d5-4558-8c01-54959e005a4e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.417683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce7474-0fc2-4984-82c6-7bce40f37ac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.419138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce7474-0fc2-4984-82c6-7bce40f37ac1 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.480062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.523710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c1330e-35d5-4558-8c01-54959e005a4e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.525546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 315/643] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:36.526251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce7474-0fc2-4984-82c6-7bce40f37ac1 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.568473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.569638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.570084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.570363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.571469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:36.571784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.573926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.579186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.589362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 329/644] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:36.594855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.643077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce7474-0fc2-4984-82c6-7bce40f37ac1 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.677825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20961b7-cd68-4468-ab01-4136c475f94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.679365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20961b7-cd68-4468-ab01-4136c475f94c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.679684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20961b7-cd68-4468-ab01-4136c475f94c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.679960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20961b7-cd68-4468-ab01-4136c475f94c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.689956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20961b7-cd68-4468-ab01-4136c475f94c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-e20961b7-cd68-4468-ab01-4136c475f94c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:36.698704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:36.699703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:36.700598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 316/645] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:36.701314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20961b7-cd68-4468-ab01-4136c475f94c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.706326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3646d8-1aec-47d7-9a28-0a2635385351 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.727908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.746404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841450-c55f-4eae-869c-8787285847de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.747199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841450-c55f-4eae-869c-8787285847de None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.747480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841450-c55f-4eae-869c-8787285847de None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.747745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841450-c55f-4eae-869c-8787285847de None admin] PATH_INFO: `/v3/roles/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.759190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07841450-c55f-4eae-869c-8787285847de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-07841450-c55f-4eae-869c-8787285847de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:36.769167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:36.770092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:36.771069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:36.775576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 330/646] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:36.776434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841450-c55f-4eae-869c-8787285847de None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.814136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20961b7-cd68-4468-ab01-4136c475f94c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.817281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:36.851699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.852455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.852776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.853059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.863304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.864922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.880592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841450-c55f-4eae-869c-8787285847de None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:36.884798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:36.885717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:36.886602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:36.886602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:36.886602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:36.886602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:36.891388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 317/647] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:36.892094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.917546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:36.918350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:36.918633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:36.918910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:36.926000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:36.927429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:36.942264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:36.943188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 331/648] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:36.945184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:36.989772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.006915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a75bb9-3446-45f6-ae4a-33ef2fb2d6e1 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.044060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.045072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.045365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.045640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.046391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.046708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.048091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.056536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 318/649] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.057342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.058577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98c9ae-85c8-4d66-8e28-9749ed604784 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.080647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.100276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.100934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.101218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.101526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.102305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] RBAC: Adding query filter params (name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.102563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.104102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.113311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 332/650] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.119028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.165606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863c1b1-e13a-47f3-a00b-767d3940e8e8 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.205050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.206045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.206335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.206614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.207342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.207652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.209080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.221495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 319/651] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.222620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.232872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.236156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab28e3-7b06-4e7e-8de0-41f198fc0b81 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.271155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.271820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.272742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.273023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.273877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.274191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.275647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.291269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 333/652] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.292105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.314298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.336559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713950c5-1ea1-4386-a1d2-7127ba44b6aa None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.372864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.373594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.373880 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.374159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.385201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.386674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:37.404854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:37.405771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:37.406653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:37.406653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:37.406653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:37.406653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:37.406653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 320/653] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:37.406923 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.413759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a979a-1f50-4303-8b2f-9ae835528edb None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.450890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.451687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.451973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.452287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.463486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.464981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:37.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:37.483936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:37.489750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:37.489750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:37.489750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:37.489750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:37.489750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 334/654] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:37.489750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.504153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.518497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4cfeac-ca62-47a8-be06-8128016bc501 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.548317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.554934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.555731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.556017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.556323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.563148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.564624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:37.579768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:37.580864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 321/655] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:37.581960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.594472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946627d-738c-4d7a-b0aa-283833dafb6c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.629242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.629821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.630108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.630379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.631227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=e6a0f84bf3f245bdab1391a2451cb4c0) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.631762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.633255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.639048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 335/656] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/role_assignments?user.id=e6a0f84bf3f245bdab1391a2451cb4c0&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.639657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.690036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e669a-e84d-46ae-8845-ecc4e52bfb35 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.730801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c87260-bf8a-4e6e-9f6a-61e2c151846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.731487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c87260-bf8a-4e6e-9f6a-61e2c151846a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.731771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c87260-bf8a-4e6e-9f6a-61e2c151846a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.732048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c87260-bf8a-4e6e-9f6a-61e2c151846a None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.732845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c87260-bf8a-4e6e-9f6a-61e2c151846a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.733159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c87260-bf8a-4e6e-9f6a-61e2c151846a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.733700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.734555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c87260-bf8a-4e6e-9f6a-61e2c151846a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.751895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 322/657] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.751895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd01155-5bf0-4770-8d08-67946793fdb7 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.751895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd01155-5bf0-4770-8d08-67946793fdb7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.751895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd01155-5bf0-4770-8d08-67946793fdb7 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.751895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 323/658] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:37.751895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd01155-5bf0-4770-8d08-67946793fdb7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.771728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb689e-c254-4964-8ed0-3c6f511aced5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.813441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.814117 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.814408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.814688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.815429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.815743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.817191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.818381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.834964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 336/659] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.835865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:37.885713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd01155-5bf0-4770-8d08-67946793fdb7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.924354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ffea2-a955-4a29-a977-efb96005e1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.925028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ffea2-a955-4a29-a977-efb96005e1d0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.925315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ffea2-a955-4a29-a977-efb96005e1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.925593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ffea2-a955-4a29-a977-efb96005e1d0 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.926382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ffea2-a955-4a29-a977-efb96005e1d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:37.926699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ffea2-a955-4a29-a977-efb96005e1d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.928182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ffea2-a955-4a29-a977-efb96005e1d0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:37.940291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 324/660] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:37.941151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688dd96-d78a-4b6c-ba20-811ec0d9cddb None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.941608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688dd96-d78a-4b6c-ba20-811ec0d9cddb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.941739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688dd96-d78a-4b6c-ba20-811ec0d9cddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.944647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0686c-c216-4975-bf99-74c8fa06fe03 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:37.960575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4688dd96-d78a-4b6c-ba20-811ec0d9cddb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:37.961863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4688dd96-d78a-4b6c-ba20-811ec0d9cddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:37.982747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:37.984394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:37.984991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:37.985452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:37.989335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:37.995923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:37.997609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:38.017706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:38.018738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 25 05:41:38.019665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:38.022925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 337/661] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:38.024351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.033452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 325/662] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:38.035105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688dd96-d78a-4b6c-ba20-811ec0d9cddb None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.094268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.133090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57554d2-59b0-4d3a-a360-9e0c7d2816ac None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.177868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688dd96-d78a-4b6c-ba20-811ec0d9cddb None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.178093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.178093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.178093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.178093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.181306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.182811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.189970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.210224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.210224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.210224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.210224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=f8a47b60be1a45f9aabc262b2d04d596) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:38.210224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.210224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:38.210686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:38.211704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 25 05:41:38.212746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:38.213320 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 326/663] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/role_assignments?user.id=f8a47b60be1a45f9aabc262b2d04d596&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:38.213400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 338/664] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:38.213474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.213548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.283968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.337765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ac667-b791-4671-906a-e400e30fa558 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.356742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0a819-1a54-47ac-b19c-a19adb4089b5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.377000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed18ce8-3f74-4b36-8e01-fcf2318ababd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.379224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed18ce8-3f74-4b36-8e01-fcf2318ababd None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.379224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed18ce8-3f74-4b36-8e01-fcf2318ababd None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.379224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed18ce8-3f74-4b36-8e01-fcf2318ababd None admin] PATH_INFO: `/v3/roles/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.391329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed18ce8-3f74-4b36-8e01-fcf2318ababd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:38.398316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.398996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.399279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.399555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-8ed18ce8-3f74-4b36-8e01-fcf2318ababd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:38.400339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:38.401261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:38.402486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:38.403189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:38.403189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.403333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 339/665] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:38.403402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.403474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed18ce8-3f74-4b36-8e01-fcf2318ababd None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.425127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 327/666] 10.222.0.40 () {56 vars in 1139 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:38.426004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.481178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.541742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed18ce8-3f74-4b36-8e01-fcf2318ababd None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.549374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b8f1c-236c-49ff-821a-97b95e5df46d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.565743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.589980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.591467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.592232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.592529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.593384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:38.593701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.595315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.596690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.597705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.598925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.599388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.600934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:38.601548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.603928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.614756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 328/667] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:38.617863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.620240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 340/668] 10.222.0.40 () {56 vars in 1137 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:38.621645 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.651209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.733032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b599f-f908-43db-a762-1166f91fa2f4 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.734942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37797db5-0652-46f5-b0b5-1ccd6c0b728c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.770600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.773425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.773425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.774026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.783429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b906c0-2c5d-4760-8771-5394c7f22362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.784089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b906c0-2c5d-4760-8771-5394c7f22362 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.784392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b906c0-2c5d-4760-8771-5394c7f22362 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.784667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b906c0-2c5d-4760-8771-5394c7f22362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:38.784843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.786303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:38.807826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:38.809103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:38.810083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:38.810083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:38.810083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:38.810083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:38.810083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 341/669] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:38.810376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:38.839385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.867219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b906c0-2c5d-4760-8771-5394c7f22362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:38.869450 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b906c0-2c5d-4760-8771-5394c7f22362 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:38.904210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:38.919069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f464aa-dab7-46d3-8149-eee4a743cbe7 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:38.957458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccacfd-cf97-4390-bf07-01a1b262cc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:38.958106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccacfd-cf97-4390-bf07-01a1b262cc33 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:38.958402 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccacfd-cf97-4390-bf07-01a1b262cc33 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:38.958660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccacfd-cf97-4390-bf07-01a1b262cc33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.011028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 329/670] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:39.012658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b906c0-2c5d-4760-8771-5394c7f22362 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.035747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccacfd-cf97-4390-bf07-01a1b262cc33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.037406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccacfd-cf97-4390-bf07-01a1b262cc33 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.067964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:39.132640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b906c0-2c5d-4760-8771-5394c7f22362 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.146199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 342/671] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:39.147934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccacfd-cf97-4390-bf07-01a1b262cc33 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.157490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:39.168755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.169484 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.169769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.173249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.173249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:39.173249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.173249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.186263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 330/672] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:39.187146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.251880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccacfd-cf97-4390-bf07-01a1b262cc33 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.284303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:39.294997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.298757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.298757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.298757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.305690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153637f-32bd-4d13-aa87-6ab58295880d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.308814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.310491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:39.331228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:39.332273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:39.333177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:39.333177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:39.333177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:39.333177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:39.336167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 343/673] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:39.337177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.356172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.357300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.357589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.357867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.372957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.375259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:39.399115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:39.400075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:39.401019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:39.401019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:39.401019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:39.401019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:39.401019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 331/674] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:39.401291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.444310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:39.455697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c9b2a4-39f4-4d1f-88e8-2d7f86dc5f6f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.496461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.497691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.498583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.499109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.510660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.512487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.526298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c93f1-354b-4fc6-bbca-29d1d0f77057 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:39.531992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:39.532951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:39.533879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:39.537311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 344/675] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:39.538900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.571338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.572005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.572728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.573029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.573901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:39.574398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.576051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.591748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:39.596569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 332/676] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:39.597500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.671251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e8f59-0303-434e-ac71-9b32a726d898 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.709715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.710523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.710767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.711052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.711785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:39.712378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.713614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.722611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:39.727872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 345/677] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:39.728961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cce674-9017-47a9-b72d-57952a62aa62 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.729132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.767385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.768250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.768640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.769027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.770082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:39.771139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.772435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.792959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 333/678] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:39.793896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.838621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:39.844056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe85461-2366-4cc4-a24c-209a8948f16c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.883936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.885107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.885672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.886092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.893418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.895209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.900381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59bc2ef-8e98-4e9a-98cd-2d2fb2fd132b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:39.910788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:39.911872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:39.913561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:39.913561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:39.913561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:39.913561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:39.913561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:39.913561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:39.913561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:39.913561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:39.917434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 346/679] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:39.918158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.938250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa679bd-027c-410e-adb0-bef99c9ee0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:39.939054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa679bd-027c-410e-adb0-bef99c9ee0a2 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.939336 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa679bd-027c-410e-adb0-bef99c9ee0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.939612 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa679bd-027c-410e-adb0-bef99c9ee0a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.946857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa679bd-027c-410e-adb0-bef99c9ee0a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:39.948332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa679bd-027c-410e-adb0-bef99c9ee0a2 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-caa679bd-027c-410e-adb0-bef99c9ee0a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:39.962628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:39.963730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:39.964680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 334/680] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:39.965248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5273c16-bc9d-44be-ba79-056b3ef58be6 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:39.965355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5273c16-bc9d-44be-ba79-056b3ef58be6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:39.965642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5273c16-bc9d-44be-ba79-056b3ef58be6 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:39.967038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 335/681] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:39.967578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5273c16-bc9d-44be-ba79-056b3ef58be6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:39.985089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.040522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdda42f-a757-4fb7-aefb-85b0ed249393 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.076291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a85c4c-59bd-4270-822a-a6c33472213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.076874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a85c4c-59bd-4270-822a-a6c33472213c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.077163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a85c4c-59bd-4270-822a-a6c33472213c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.077456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a85c4c-59bd-4270-822a-a6c33472213c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.080822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5273c16-bc9d-44be-ba79-056b3ef58be6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.106236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.117963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.118942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.119231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.119513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.120342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:40.120713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:40.122231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:40.134570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 336/682] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:40.135391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.155284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a85c4c-59bd-4270-822a-a6c33472213c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:40.156899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a85c4c-59bd-4270-822a-a6c33472213c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:40.217119 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.244053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df714527-e0ec-4082-8207-3703183d9d53 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.261941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 347/683] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:40.263315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88c79-a58f-46f3-a4d8-58e36cb527a2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.263601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88c79-a58f-46f3-a4d8-58e36cb527a2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.263875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b88c79-a58f-46f3-a4d8-58e36cb527a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.281212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-52b88c79-a58f-46f3-a4d8-58e36cb527a2 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:40.282416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-52b88c79-a58f-46f3-a4d8-58e36cb527a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:40.284776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.285520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.285877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.286163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.286955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:40.287268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:40.289396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:40.302119 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 337/684] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:40.302768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.343544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.368145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 348/685] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:40.368145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b88c79-a58f-46f3-a4d8-58e36cb527a2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.424435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9167f4-3c18-4d03-a1e6-5d71b6b0c1a5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.460155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.460888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.461256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.461593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.462483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=b1638c28af8240439d061a3ae0ddc38d) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:40.462593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b88c79-a58f-46f3-a4d8-58e36cb527a2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.462914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:40.464453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:40.466331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.470757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 338/686] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/role_assignments?user.id=b1638c28af8240439d061a3ae0ddc38d&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:40.471528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.502094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.502915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.503199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.503476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.515326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:40.516846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:40.538017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:40.539008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:40.539897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:40.539897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:40.539897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:40.539897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:40.539897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 349/687] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:40.540849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.588210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.606071 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfc732-159c-43f8-ac4e-4a875a34f62f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.648971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9b696-0619-42d2-a8d3-4737b25ca481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.649629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b696-0619-42d2-a8d3-4737b25ca481 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.649951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b696-0619-42d2-a8d3-4737b25ca481 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.650236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b696-0619-42d2-a8d3-4737b25ca481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.651069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9b696-0619-42d2-a8d3-4737b25ca481 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=3c0883bc26a04de2916c78df342d2023) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:40.651391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9b696-0619-42d2-a8d3-4737b25ca481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:40.652861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9b696-0619-42d2-a8d3-4737b25ca481 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:40.659018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 339/688] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/role_assignments?user.id=3c0883bc26a04de2916c78df342d2023&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:40.659465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dd29ac-7a57-44a4-bb17-7b3abdcb5d86 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.660294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467039b-a2e7-423f-b23a-56746c901e9e None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.660587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467039b-a2e7-423f-b23a-56746c901e9e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.660869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467039b-a2e7-423f-b23a-56746c901e9e None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.667117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 340/689] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:40.668102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467039b-a2e7-423f-b23a-56746c901e9e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.708872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61aeba9-f59e-444d-9ccc-7ad544522247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.710896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aeba9-f59e-444d-9ccc-7ad544522247 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.711447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aeba9-f59e-444d-9ccc-7ad544522247 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.711936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61aeba9-f59e-444d-9ccc-7ad544522247 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.724676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61aeba9-f59e-444d-9ccc-7ad544522247 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-a61aeba9-f59e-444d-9ccc-7ad544522247 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:40.733230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:40.734243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:40.735090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:40.737693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 350/690] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:40.739345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15483e-56be-4ea2-a80f-302e99061231 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.739925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15483e-56be-4ea2-a80f-302e99061231 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.740535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad15483e-56be-4ea2-a80f-302e99061231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.755449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.761219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ad15483e-56be-4ea2-a80f-302e99061231 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:40.762766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ad15483e-56be-4ea2-a80f-302e99061231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:40.811899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467039b-a2e7-423f-b23a-56746c901e9e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.849853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 351/691] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:40.850728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15483e-56be-4ea2-a80f-302e99061231 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.864020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:40.864655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:40.865053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:40.865333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:40.866878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:40.867190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:40.868714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:40.882823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 341/692] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:40.883918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:40.890931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:40.978491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad15483e-56be-4ea2-a80f-302e99061231 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:40.998814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e251f1c-3feb-44bb-a26e-5fcdfcd9ea11 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.026384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.034230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.035451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.035887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.036333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.041892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:41.041892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.041892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.042187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.042187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.042187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.042187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.042586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.043238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.050274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 352/693] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.051299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:41.065856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:41.067113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:41.068152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:41.072291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 342/694] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Jul 25 05:41:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:41.073331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.167450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.177662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729a0d7c-3c7d-4ff3-aa1b-fe7672c5d062 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.189181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa54ce6-aa76-4e18-a7e1-29c6d3f05c24 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.217713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.218668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.219068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.219438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.225477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.226283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.226572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.226856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.227254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.228903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.238101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.239587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:41.244428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:41.245318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:41.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:41.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:41.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:41.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:41.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:41.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:41.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:41.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:41.250540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 353/695] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:41.251435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:41.259492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:41.261552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:41.263677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:41.263677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:41.263677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:41.263677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:41.264206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 343/696] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:41.272165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.308878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.364027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99157b-560f-40f0-8a38-9b86f20c301e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.372969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10afa67-e93b-4883-b97a-c6038506026b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.401466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.402154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.402437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.402713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.403435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:41.403740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.405191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.415660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 354/697] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.415660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.441380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.530888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.531936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.532445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.532946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.533956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:41.534462 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e5749-7d98-4d91-b96b-5444d5c30205 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.534656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.536322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.550425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 344/698] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.551645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.579454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.580142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.580437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.580713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.581446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:41.581754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.583232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.600840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 355/699] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.600840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.670854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.675068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2c8cc-911c-4388-abf8-2db8a025fe9c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.697995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:41.713962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.714930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.715762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.715762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.716822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:41.717249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.718698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.727660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a965fd-c2b2-42a3-b13d-a581990d7839 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.728480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 345/700] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.729473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:41.810141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.811373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.812564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.819560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.819560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=cf71e1ef5acb429b96219888830181d3) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:41.819560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.823065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.841411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 356/701] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/role_assignments?user.id=cf71e1ef5acb429b96219888830181d3&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:41.906454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11b73c4-cacb-4465-bb02-dec798d7645a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:41.947783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:41.948721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:41.949028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:41.949307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:41.957945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:41.960265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:41.977778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:41.980019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:41.982307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 346/702] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:41.988141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:42.104277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a254349c-49d7-4d50-8370-157957c75748 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:42.139328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:42.140008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:42.140327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:42.140608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:42.141458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:42.141769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:42.143207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:42.153130 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:42.161984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 357/703] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:42.168053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:42.201378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:42.284076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f199c-375e-4ec9-8cbc-0167dd340f82 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:42.322042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f60e76-ebd7-4924-bf4b-d866431f9fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:42.322711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f60e76-ebd7-4924-bf4b-d866431f9fe6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:42.322993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f60e76-ebd7-4924-bf4b-d866431f9fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:42.323268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f60e76-ebd7-4924-bf4b-d866431f9fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:42.396102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f60e76-ebd7-4924-bf4b-d866431f9fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:42.398399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f60e76-ebd7-4924-bf4b-d866431f9fe6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:42.422873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:42.505954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 347/704] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:42.511425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:42.633933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805d785-240d-470c-8bfb-3d75da8675c2 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:42.680054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a1759-5ef2-4ef3-9b6e-34783ccbbd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:42.680971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a1759-5ef2-4ef3-9b6e-34783ccbbd34 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:42.681256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a1759-5ef2-4ef3-9b6e-34783ccbbd34 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:42.681532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a1759-5ef2-4ef3-9b6e-34783ccbbd34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:42.692532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a1759-5ef2-4ef3-9b6e-34783ccbbd34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:42.695050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a1759-5ef2-4ef3-9b6e-34783ccbbd34 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-e20a1759-5ef2-4ef3-9b6e-34783ccbbd34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:42.722501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:42.723484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:42.724453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:42.724453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:42.724453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:42.724453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:42.724453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 358/705] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:42.729828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f60e76-ebd7-4924-bf4b-d866431f9fe6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:42.739313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14180e-c740-4dd0-ad1e-60f1bdd90d60 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:42.739615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14180e-c740-4dd0-ad1e-60f1bdd90d60 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:42.739870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14180e-c740-4dd0-ad1e-60f1bdd90d60 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:42.743879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 359/706] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:42.750325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93fab8-1bb6-446a-b431-2cea7672d210 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:42.750614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93fab8-1bb6-446a-b431-2cea7672d210 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:42.750898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93fab8-1bb6-446a-b431-2cea7672d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:42.790110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-bb93fab8-1bb6-446a-b431-2cea7672d210 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:42.791322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bb93fab8-1bb6-446a-b431-2cea7672d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:42.884324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f60e76-ebd7-4924-bf4b-d866431f9fe6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:42.909433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:42.922962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:42.923637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:42.923921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:42.924398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:42.925131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:42.925440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:42.926918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:42.930959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:42.942156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 348/707] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:42.948622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:42.998072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 360/708] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:43.033232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93fab8-1bb6-446a-b431-2cea7672d210 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:43.078729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e1722-0e64-4baf-95c9-74f954c8dec2 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:43.116015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688cf3a-ad67-48ad-a510-ba6ba7b5c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:43.117207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688cf3a-ad67-48ad-a510-ba6ba7b5c440 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.117547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688cf3a-ad67-48ad-a510-ba6ba7b5c440 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.117926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688cf3a-ad67-48ad-a510-ba6ba7b5c440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.125550 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688cf3a-ad67-48ad-a510-ba6ba7b5c440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:43.127044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688cf3a-ad67-48ad-a510-ba6ba7b5c440 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:43.137746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93fab8-1bb6-446a-b431-2cea7672d210 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b688cf3a-ad67-48ad-a510-ba6ba7b5c440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:43.142557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:43.143459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:43.144369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 349/709] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:43.145423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accd7a8-0129-44f9-8ffa-099948272271 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.145708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accd7a8-0129-44f9-8ffa-099948272271 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.145983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accd7a8-0129-44f9-8ffa-099948272271 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.152048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 350/710] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:43.152918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accd7a8-0129-44f9-8ffa-099948272271 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:43.182154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79ee59-d0ba-4368-8003-f07470c101d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:43.183065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ee59-d0ba-4368-8003-f07470c101d8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.183362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ee59-d0ba-4368-8003-f07470c101d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.183637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79ee59-d0ba-4368-8003-f07470c101d8 None admin] PATH_INFO: `/v3/services/network` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.184327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79ee59-d0ba-4368-8003-f07470c101d8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:43.185943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79ee59-d0ba-4368-8003-f07470c101d8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-7e79ee59-d0ba-4368-8003-f07470c101d8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:43.201392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:43.203716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:43.205727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:43.207742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 361/711] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:43 2022] GET /identity/v3/services/network => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:43.212073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:43.212680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011b51-32cd-410a-b19b-98521e76a0b2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.212680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011b51-32cd-410a-b19b-98521e76a0b2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.212680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54011b51-32cd-410a-b19b-98521e76a0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.227467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-54011b51-32cd-410a-b19b-98521e76a0b2 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:43.228690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-54011b51-32cd-410a-b19b-98521e76a0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:43.267749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accd7a8-0129-44f9-8ffa-099948272271 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:43.273868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:43.308235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 362/712] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:43.308235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54011b51-32cd-410a-b19b-98521e76a0b2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:43.313807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:43.314798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.315359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.315869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.316892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:43.317344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:43.320190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:43.346946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 351/713] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:43.356796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:43.413707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54011b51-32cd-410a-b19b-98521e76a0b2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:43.451113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:43.451793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.452078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.452387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.453526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] RBAC: Adding query filter params (name=network) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:43.453526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:43.454938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:43.464964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 363/714] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:43.465926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:43.467767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994287ce-5b17-4583-9951-2ed256d2c180 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:43.470548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:43.507206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:43.508101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.508471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.508752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.510258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:43.520899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:43.523369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:43.543265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:43.544446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:43.545555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:43.545555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:43.545555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:43.545555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:43.549607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 352/715] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:43.550567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:43.578703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff1380-48e9-4831-89d8-d99cc72057ae None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:43.616611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:43.617523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.617957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.618370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.619486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=e6a0f84bf3f245bdab1391a2451cb4c0) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:43.619988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:43.621746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:43.635142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 364/716] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:43 2022] GET /identity/v3/role_assignments?user.id=e6a0f84bf3f245bdab1391a2451cb4c0&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:43.636545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:43.668440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a89ad-b57c-482d-841f-dfa06a1a5976 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:43.705212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:43.705902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.706187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.706459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.707181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] RBAC: Adding query filter params (type=network) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:43.707485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:43.709914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:43.715656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 353/717] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:43.832261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6e65e6-f2bd-42ea-a7b3-566b4c5488e1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:43.833797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:43.871923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a90b49-800d-432e-8e7a-72e7c34b47ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:43.872668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a90b49-800d-432e-8e7a-72e7c34b47ef None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:43.872952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a90b49-800d-432e-8e7a-72e7c34b47ef None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:43.873230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a90b49-800d-432e-8e7a-72e7c34b47ef None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:43.873985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a90b49-800d-432e-8e7a-72e7c34b47ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:43.874294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a90b49-800d-432e-8e7a-72e7c34b47ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:43.875729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a90b49-800d-432e-8e7a-72e7c34b47ef None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:43.890344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 365/718] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:43.899054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:43.923783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:44.033102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc88c2-b662-43db-8b97-321d827a7f22 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:44.072948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74b1ea-5125-4650-a4c3-412fc11ba766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:44.073643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b1ea-5125-4650-a4c3-412fc11ba766 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:44.073931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b1ea-5125-4650-a4c3-412fc11ba766 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:44.074213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b1ea-5125-4650-a4c3-412fc11ba766 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:44.074929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74b1ea-5125-4650-a4c3-412fc11ba766 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:44.076400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74b1ea-5125-4650-a4c3-412fc11ba766 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:44.134237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda3eb-43d7-491f-a197-5b7f31028f84 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:44.134739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda3eb-43d7-491f-a197-5b7f31028f84 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:44.134739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cda3eb-43d7-491f-a197-5b7f31028f84 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:44.136537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 366/719] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:44.141632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3da88-0013-41eb-80cb-b7ace8a082c6 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:44.141920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3da88-0013-41eb-80cb-b7ace8a082c6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:44.142197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3da88-0013-41eb-80cb-b7ace8a082c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:44.159855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d1e3da88-0013-41eb-80cb-b7ace8a082c6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:44.161210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d1e3da88-0013-41eb-80cb-b7ace8a082c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:44.170344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 354/720] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:41:43 2022] POST /identity/v3/users => generated 254 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:41:44.262592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 367/721] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:41:44.297530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74b1ea-5125-4650-a4c3-412fc11ba766 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:44.400358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:44.431976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74b1ea-5125-4650-a4c3-412fc11ba766 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:44.468521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545bbe31-521e-40ea-81b2-ec6fe0877fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:44.469252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bbe31-521e-40ea-81b2-ec6fe0877fe1 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:44.469597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bbe31-521e-40ea-81b2-ec6fe0877fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:44.469847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545bbe31-521e-40ea-81b2-ec6fe0877fe1 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:44.470600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bbe31-521e-40ea-81b2-ec6fe0877fe1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:44.471985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545bbe31-521e-40ea-81b2-ec6fe0877fe1 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:44.552897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 355/722] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:44 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:44.899265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ad05-b34b-451b-a772-5aae33b82c22 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:44.899634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ad05-b34b-451b-a772-5aae33b82c22 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:44.899832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33ad05-b34b-451b-a772-5aae33b82c22 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:44.901519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 368/723] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:44.906921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06a91-bde4-4f7d-843f-7618916110b7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:44.907217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06a91-bde4-4f7d-843f-7618916110b7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:44.907505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06a91-bde4-4f7d-843f-7618916110b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:44.928532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-48c06a91-bde4-4f7d-843f-7618916110b7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:44.929762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-48c06a91-bde4-4f7d-843f-7618916110b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:45.022872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 356/724] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 25 05:41:45.045777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33ad05-b34b-451b-a772-5aae33b82c22 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:45.169940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:45.183235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c33ad05-b34b-451b-a772-5aae33b82c22 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:45.227405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c432a-ccd0-4eca-8e98-36623f9da008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:45.228283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c432a-ccd0-4eca-8e98-36623f9da008 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:45.228571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c432a-ccd0-4eca-8e98-36623f9da008 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:45.228852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c432a-ccd0-4eca-8e98-36623f9da008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:45.245155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c432a-ccd0-4eca-8e98-36623f9da008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-b56c432a-ccd0-4eca-8e98-36623f9da008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:45.259279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:45.260247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:45.261071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 369/725] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:45.268021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c06a91-bde4-4f7d-843f-7618916110b7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:45.354484 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:45.410604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c06a91-bde4-4f7d-843f-7618916110b7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:45.459670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:45.460489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:45.460778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:45.461056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:45.461819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:45.462130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:45.463565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:45.476262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 357/726] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:45.481751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c432a-ccd0-4eca-8e98-36623f9da008 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:45.603944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c432a-ccd0-4eca-8e98-36623f9da008 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:45.648960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:45.649692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:45.649980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:45.650259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:45.666764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:45.668575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:45.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:45.689695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:45.690636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:45.690636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:45.690636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:45.690636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:45.695376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 370/727] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:45.701794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:45.899310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff27d60-e478-4593-83ed-614e91e6ae94 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:45.930676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:45.949696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6d2c6-8733-4db9-bb85-c2f487db66bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:45.950374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6d2c6-8733-4db9-bb85-c2f487db66bc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:45.950656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6d2c6-8733-4db9-bb85-c2f487db66bc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:45.950932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6d2c6-8733-4db9-bb85-c2f487db66bc None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:45.951656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6d2c6-8733-4db9-bb85-c2f487db66bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:45.951963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6d2c6-8733-4db9-bb85-c2f487db66bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:45.953347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc6d2c6-8733-4db9-bb85-c2f487db66bc None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:45.966259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 358/728] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:45.971751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:45.992450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:46.170190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7e55d-bb9d-4606-9b4b-27cae3b22d3c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:46.197776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8abf-0e01-41d4-89bd-b05c63c19e72 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.198108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8abf-0e01-41d4-89bd-b05c63c19e72 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.198343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d8abf-0e01-41d4-89bd-b05c63c19e72 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.199732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 359/729] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:46.209723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7c309-602d-49b7-85bc-cc269b340dde None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.210125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7c309-602d-49b7-85bc-cc269b340dde None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.210405 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e7c309-602d-49b7-85bc-cc269b340dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.218961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:46.219763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.220051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.220364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.228492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:46.229958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:46.245191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-34e7c309-602d-49b7-85bc-cc269b340dde None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:46.246392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-34e7c309-602d-49b7-85bc-cc269b340dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:46.252758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:46.253718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:46.254653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 371/730] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:46.262565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:46.332539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:46.376478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444c707-7536-42cd-b683-c181942ddfb5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:46.377694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 360/731] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:46.379328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b51a5d-99c4-4f97-be34-3527d875b016 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.379621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b51a5d-99c4-4f97-be34-3527d875b016 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.379902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b51a5d-99c4-4f97-be34-3527d875b016 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.383109 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 361/732] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:46.393570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7205d-b325-441b-ad1f-8592a9b87b21 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.393990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7205d-b325-441b-ad1f-8592a9b87b21 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.394372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7205d-b325-441b-ad1f-8592a9b87b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.418998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:46.419262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a6b7205d-b325-441b-ad1f-8592a9b87b21 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:46.420152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.420399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a6b7205d-b325-441b-ad1f-8592a9b87b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:46.420871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.421300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.422473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:46.422958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:46.424680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:46.444788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 372/733] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 25 05:41:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:46.446417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:46.496373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 362/734] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 25 05:41:46.497283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7205d-b325-441b-ad1f-8592a9b87b21 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:46.559839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:46.570486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf64b88f-499e-48bd-9ebf-b262fe8cfaa8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:46.604266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:46.620893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7205d-b325-441b-ad1f-8592a9b87b21 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:46.624131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc82f-209c-4556-8596-d75082ed1c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:46.625117 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc82f-209c-4556-8596-d75082ed1c69 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.625580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc82f-209c-4556-8596-d75082ed1c69 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.625995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbc82f-209c-4556-8596-d75082ed1c69 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.626884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbc82f-209c-4556-8596-d75082ed1c69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:46.628432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbc82f-209c-4556-8596-d75082ed1c69 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:46.657262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15b463-626a-40ee-922b-8f82f374060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:46.658040 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15b463-626a-40ee-922b-8f82f374060b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.658327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15b463-626a-40ee-922b-8f82f374060b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.658604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15b463-626a-40ee-922b-8f82f374060b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.708682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 373/735] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:46 2022] POST /identity/v3/services => generated 240 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:46.709985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc82f-209c-4556-8596-d75082ed1c69 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:46.751248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15b463-626a-40ee-922b-8f82f374060b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:46.752862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15b463-626a-40ee-922b-8f82f374060b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:46.848581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbc82f-209c-4556-8596-d75082ed1c69 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:46.870748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:46.895396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28db838c-05cd-4a2c-b9b9-53ebf146f795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:46.897741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db838c-05cd-4a2c-b9b9-53ebf146f795 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.898457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db838c-05cd-4a2c-b9b9-53ebf146f795 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.899321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db838c-05cd-4a2c-b9b9-53ebf146f795 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.916184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28db838c-05cd-4a2c-b9b9-53ebf146f795 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-28db838c-05cd-4a2c-b9b9-53ebf146f795 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:46.928427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:46.929456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:46.930287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:46.933803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 374/736] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:46.935356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ecb6-e5d8-456f-b4da-0bb238c82c5a None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.935774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ecb6-e5d8-456f-b4da-0bb238c82c5a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.936208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08ecb6-e5d8-456f-b4da-0bb238c82c5a None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.937867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 375/737] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:46.939852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08ecb6-e5d8-456f-b4da-0bb238c82c5a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:46.953998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 363/738] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:46 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 25 05:41:46.955456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b319f69-e04a-4a2f-a738-3a3c7d97df7e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:46.955751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b319f69-e04a-4a2f-a738-3a3c7d97df7e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:46.956030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b319f69-e04a-4a2f-a738-3a3c7d97df7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:46.976632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-6b319f69-e04a-4a2f-a738-3a3c7d97df7e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:46.977757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-6b319f69-e04a-4a2f-a738-3a3c7d97df7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:46.995637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.068249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 364/739] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:47.070168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b319f69-e04a-4a2f-a738-3a3c7d97df7e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.077027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08ecb6-e5d8-456f-b4da-0bb238c82c5a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.122159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.123247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.123727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.124307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.125343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:47.125988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.127633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.137497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 376/740] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:47.138632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.176519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.221593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b319f69-e04a-4a2f-a738-3a3c7d97df7e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.257029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.258883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.259973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.260479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.260706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0d790-8415-4d5a-83f0-d5df7c437466 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.261157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.273270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.275260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:47.294333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:47.295261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:47.296204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:47.296204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:47.296204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:47.296204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:47.302155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 365/741] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:47.302155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.305319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58b194-125a-4bbc-b627-d1425eedb86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.307073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58b194-125a-4bbc-b627-d1425eedb86c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.307073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58b194-125a-4bbc-b627-d1425eedb86c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.307073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58b194-125a-4bbc-b627-d1425eedb86c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.319305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58b194-125a-4bbc-b627-d1425eedb86c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-5c58b194-125a-4bbc-b627-d1425eedb86c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:47.328052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:47.329048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:47.329903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:47.333425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 377/742] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:47.335113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58b194-125a-4bbc-b627-d1425eedb86c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.419509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6858ce6f-3491-4385-887a-4717b21e360a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.444511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.447016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58b194-125a-4bbc-b627-d1425eedb86c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.455145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b14d99-b576-4e09-b445-fd5f8e80f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.457157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14d99-b576-4e09-b445-fd5f8e80f584 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.457157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14d99-b576-4e09-b445-fd5f8e80f584 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.457157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14d99-b576-4e09-b445-fd5f8e80f584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.467471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b14d99-b576-4e09-b445-fd5f8e80f584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.468976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b14d99-b576-4e09-b445-fd5f8e80f584 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.482204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.482870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.483154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.483429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.484180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:47.484491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.485863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b5b14d99-b576-4e09-b445-fd5f8e80f584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:47.487607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:47.488496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:47.489422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:47.489422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:47.489422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:47.489422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:47.489422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 366/743] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:47.491251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22680e-51d9-4ee7-a1ad-7ed1834a7ed2 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.491251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22680e-51d9-4ee7-a1ad-7ed1834a7ed2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.491561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f22680e-51d9-4ee7-a1ad-7ed1834a7ed2 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.496292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 367/744] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:47.496292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22680e-51d9-4ee7-a1ad-7ed1834a7ed2 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.504368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 378/745] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:47.505207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.531408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.617875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f22680e-51d9-4ee7-a1ad-7ed1834a7ed2 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.618522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12e316-76d2-47f0-b49b-5c1b8e620030 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.655493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91c26e-5ff8-4062-a26a-99df958b2e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.655921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.656463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91c26e-5ff8-4062-a26a-99df958b2e11 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.656628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.656910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.657003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91c26e-5ff8-4062-a26a-99df958b2e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.657186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.657439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91c26e-5ff8-4062-a26a-99df958b2e11 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.657917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:47.658222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.658459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91c26e-5ff8-4062-a26a-99df958b2e11 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:47.658860 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91c26e-5ff8-4062-a26a-99df958b2e11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.659585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.660435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91c26e-5ff8-4062-a26a-99df958b2e11 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.670236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 368/746] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:47.671380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589657fc-8154-4c62-89c9-887d76754c68 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.671770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589657fc-8154-4c62-89c9-887d76754c68 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.672243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589657fc-8154-4c62-89c9-887d76754c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.681966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 379/747] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:47.681966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.695480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-589657fc-8154-4c62-89c9-887d76754c68 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:47.697073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-589657fc-8154-4c62-89c9-887d76754c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:47.727697 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.770410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:47.776534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 369/748] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:47.777592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589657fc-8154-4c62-89c9-887d76754c68 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.798610 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e29c3-755f-49e5-8a06-58831ab6f25f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.835688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.836504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.836787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.837058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.844202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.845707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:47.860589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:47.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 380/749] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:47.867762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.894166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589657fc-8154-4c62-89c9-887d76754c68 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.930481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:47.931272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:47.931573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:47.931850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:47.942348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:47.943820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:47.965925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:47.967020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:47.968046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:47.968046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:47.968046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:47.968046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:47.968465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 370/750] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:47.968908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:47.976978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf0e0d-fda4-4a45-a0c0-0290f32a79c6 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:47.994387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:48.014720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.015591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.015937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.016311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.024593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.026225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.029775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:48.042016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:48.043868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:48.045204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:48.048068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 381/751] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:48.049044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.081406 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7d398-b1e5-4365-a5b5-64d7bb4646b7 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.120408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.121374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.121658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.121934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.122585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.123944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:48.137056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:48.138016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:48.138872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 371/752] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Jul 25 05:41:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:48.139735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.160008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa3d3f-89af-4844-a8b4-c0ac8f1c445e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.197135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.197911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.198269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.198551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.199358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:48.199672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.201168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.213695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 382/753] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:48.214489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.239234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:48.258909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b22756-edb1-4509-bdbd-3ee808cd905c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.274635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:48.298036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.298835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.299275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.299598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.300455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:48.300856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.302339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.314182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 372/754] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:48.314979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.322295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eef26f5-4bdb-443b-b778-4e2fca0f7553 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.359928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.360726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.361064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.361516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.362426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:48.362798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.364435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.384616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 383/755] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:48.385446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.426984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc4f98-ecdd-4ebb-8bc5-1ef09edf4d6a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.465795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.466470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.466757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.467038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.467848 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:48.468188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.469622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.474966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 373/756] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:48.475763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.487525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:48.499811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f2094-95ee-4bf8-a560-20669daa09e0 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.518830 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:48.538527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.539430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.539714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.539985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.540855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=cf71e1ef5acb429b96219888830181d3) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:48.541173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.542642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.556235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 384/757] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/role_assignments?user.id=cf71e1ef5acb429b96219888830181d3&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:48.556235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.595374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e533e-7aed-44bb-ad53-fb42d48f20aa None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.631916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.633067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.633418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.633800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.642671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.644343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:48.664517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:48.665558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:48.666513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:48.668873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7373b-7ae9-438f-96bd-ffaebeefec0c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.670585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 374/758] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:48.671489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.715706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.716575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.717273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.717572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.731912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.734116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:48.754204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:48.755338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:48.756999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:48.756999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:48.756999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:48.756999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:48.762948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 385/759] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:48.763916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.778258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:48.808745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:48.817133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a964f-6aa9-4e9b-aa1f-516bd1e6531b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.856943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.857948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.858243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.858634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.860097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:48.860522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.861970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:48.868232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 375/760] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:48.869084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:48.937536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28193872-d435-4a2e-b221-0598f6548d9f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:48.976609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:48.977324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:48.977610 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:48.977890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:48.978729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:48.979042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:48.980521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.003394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 386/761] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.003394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.015250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0efd3-0a32-4f8d-afdd-d404bd110a5b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.046213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.051573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec504af-1586-42f7-bc24-d145d16235a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.052262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec504af-1586-42f7-bc24-d145d16235a0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.053114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec504af-1586-42f7-bc24-d145d16235a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.053398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec504af-1586-42f7-bc24-d145d16235a0 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.057913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec504af-1586-42f7-bc24-d145d16235a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:49.057913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec504af-1586-42f7-bc24-d145d16235a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.057913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec504af-1586-42f7-bc24-d145d16235a0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.069178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 376/762] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.070765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034633f9-8edc-4add-ae6a-edc388503253 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.071121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034633f9-8edc-4add-ae6a-edc388503253 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.071475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034633f9-8edc-4add-ae6a-edc388503253 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.073031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 377/763] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:49.077555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034633f9-8edc-4add-ae6a-edc388503253 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.102505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.111265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8aebb-db87-4283-b435-a617fca17a23 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.148510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.149199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.149487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.149766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.150566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] RBAC: Adding query filter params (interface=public, service_id=deafd7eff3834102b211b12a39d156c2, region_id=RegionOne) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:49.150882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.152287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.159650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 387/764] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 25 05:41:48 2022] GET /identity/v3/endpoints?service_id=deafd7eff3834102b211b12a39d156c2&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.160468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.185557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034633f9-8edc-4add-ae6a-edc388503253 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.221651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ef8d6-29c8-4784-bdcc-a122771986cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.222315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef8d6-29c8-4784-bdcc-a122771986cd None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.222595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef8d6-29c8-4784-bdcc-a122771986cd None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.222891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ef8d6-29c8-4784-bdcc-a122771986cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.266730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58d59e-dc99-4997-bfbc-5e5d486692d6 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.284182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.294195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ef8d6-29c8-4784-bdcc-a122771986cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.295639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ef8d6-29c8-4784-bdcc-a122771986cd None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.303389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc1b77-9ca1-4ed4-a3e7-327ba6218e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.304403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc1b77-9ca1-4ed4-a3e7-327ba6218e3b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.304603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc1b77-9ca1-4ed4-a3e7-327ba6218e3b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.304890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc1b77-9ca1-4ed4-a3e7-327ba6218e3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.312540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc1b77-9ca1-4ed4-a3e7-327ba6218e3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.314018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc1b77-9ca1-4ed4-a3e7-327ba6218e3b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-00bc1b77-9ca1-4ed4-a3e7-327ba6218e3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:49.329035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:49.329932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:49.330856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 388/765] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:49.332081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8186-274e-463e-9627-5fe0ea28f3a9 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.332081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8186-274e-463e-9627-5fe0ea28f3a9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.332081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d8186-274e-463e-9627-5fe0ea28f3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.348795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-6e7d8186-274e-463e-9627-5fe0ea28f3a9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:49.350745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6e7d8186-274e-463e-9627-5fe0ea28f3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:49.361456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.406661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 378/766] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:49.407777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ef8d6-29c8-4784-bdcc-a122771986cd None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.430943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 389/767] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:49.432242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d8186-274e-463e-9627-5fe0ea28f3a9 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.511496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.519843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ef8d6-29c8-4784-bdcc-a122771986cd None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.544813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d8186-274e-463e-9627-5fe0ea28f3a9 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.555281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b75d0-ad2e-4246-bf3c-4a5a04faabc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.555952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b75d0-ad2e-4246-bf3c-4a5a04faabc5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.556526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b75d0-ad2e-4246-bf3c-4a5a04faabc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.556526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b75d0-ad2e-4246-bf3c-4a5a04faabc5 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.557209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b75d0-ad2e-4246-bf3c-4a5a04faabc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.558571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b75d0-ad2e-4246-bf3c-4a5a04faabc5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.563823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 379/768] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/services => generated 772 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.564639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b75d0-ad2e-4246-bf3c-4a5a04faabc5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.588924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.589943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.590380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.590813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.591837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:49.592439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.594140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.608311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 390/769] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:49.609423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.619605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.719840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b75d0-ad2e-4246-bf3c-4a5a04faabc5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.752511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8713e85-048a-472a-a0ac-a31ce4082967 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.778840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.779639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.779916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.780218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.791875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.793364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.797725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.798650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.799034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.799412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] PATH_INFO: `/v3/services/network` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.800219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.801727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:49.814893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:49.819327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:49.820206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 391/770] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:49.821264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:49.827298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:49.828228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:49.829181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:49.829181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:49.829181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:49.829181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:49.829181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 380/771] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:49.830152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:49.845413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.910974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:49.947670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459afcb-d85b-487b-8768-d44e1d3271d7 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.953101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923007-999b-4481-bce2-bd34095336d0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:49.986290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ada69-f8dc-4e79-9d5b-af9e7840b1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.987193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ada69-f8dc-4e79-9d5b-af9e7840b1e1 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.987481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ada69-f8dc-4e79-9d5b-af9e7840b1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.987761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ada69-f8dc-4e79-9d5b-af9e7840b1e1 None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/e6a0f84bf3f245bdab1391a2451cb4c0/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:49.989800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ada69-f8dc-4e79-9d5b-af9e7840b1e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=e6a0f84bf3f245bdab1391a2451cb4c0, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:49.991290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ada69-f8dc-4e79-9d5b-af9e7840b1e1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:49.998076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:49.998745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:49.999033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:49.999314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.000024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] RBAC: Adding query filter params (name=network) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:50.000383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:50.001741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:50.006782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 381/772] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:50.007504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:50.068850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 392/773] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:49 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/e6a0f84bf3f245bdab1391a2451cb4c0/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:50.071263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ada69-f8dc-4e79-9d5b-af9e7840b1e1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:50.091573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:50.129350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ee73b-8ea4-4cce-bf97-3f9b96334a33 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:50.171316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:50.172341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.172845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.173306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.174278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:50.174877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:50.176671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:50.192029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 382/774] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:50.197644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:50.285806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ada69-f8dc-4e79-9d5b-af9e7840b1e1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:50.321435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:50.327625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:50.328387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.328745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.329101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.329910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] RBAC: Adding query filter params (type=network) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:50.330291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:50.331748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:50.340080 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 393/775] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:50.345378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:50.356785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc49cb0-021d-4ca2-bb54-21d73b8699f3 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:50.400474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:50.407851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:50.408686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.408969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.409246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.419475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:50.420967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:50.446444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:50.447420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:50.448365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:50.448365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:50.448365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:50.448365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:50.448365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:50.448365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:50.448365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:50.448365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:50.455430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 383/776] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:50.460882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:50.492574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03743dc6-f3e2-4d21-8d52-0a99ce42df23 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:50.532879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:50.534149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.534667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.535167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.536193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] RBAC: Adding query filter params (interface=public, service_id=fc7fde85ef9a4a36a6bc0c21e5ec4ed6, region_id=RegionOne) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:50.536779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:50.539294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:50.545616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 394/777] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 25 05:41:50 2022] GET /identity/v3/endpoints?service_id=fc7fde85ef9a4a36a6bc0c21e5ec4ed6&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:50.552262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:50.644478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:50.696027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9063c6-4c73-46bf-abe1-2c83b68ccc00 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:50.719223 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003a38a-e5ae-4aea-9244-d2e5b4bb6872 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:50.746971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e03cd-7aad-431b-9776-01abce5383bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:50.748068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e03cd-7aad-431b-9776-01abce5383bc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.748379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e03cd-7aad-431b-9776-01abce5383bc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.748919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e03cd-7aad-431b-9776-01abce5383bc None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.749879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e03cd-7aad-431b-9776-01abce5383bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:50.750350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e03cd-7aad-431b-9776-01abce5383bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:50.753478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e03cd-7aad-431b-9776-01abce5383bc None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:50.772138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430bc69-abdd-467d-b36d-5678dd2cdde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:50.772138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430bc69-abdd-467d-b36d-5678dd2cdde0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.772138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430bc69-abdd-467d-b36d-5678dd2cdde0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.773125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 384/778] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:50.773218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430bc69-abdd-467d-b36d-5678dd2cdde0 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.773218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430bc69-abdd-467d-b36d-5678dd2cdde0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:50.773380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1901d-af51-4ba1-924b-64598fe6ce6b None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.773380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1901d-af51-4ba1-924b-64598fe6ce6b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.773380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1901d-af51-4ba1-924b-64598fe6ce6b None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.773625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6430bc69-abdd-467d-b36d-5678dd2cdde0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:50.775292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 385/779] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:50.776470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1901d-af51-4ba1-924b-64598fe6ce6b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:50.781168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 395/780] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:41:50 2022] GET /identity/v3/services => generated 772 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:41:50.782599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6b0d2-432f-4206-831a-39a6cd7be49b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:50.782997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6b0d2-432f-4206-831a-39a6cd7be49b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:50.783383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6b0d2-432f-4206-831a-39a6cd7be49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:50.814041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-15e6b0d2-432f-4206-831a-39a6cd7be49b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:50.818721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-15e6b0d2-432f-4206-831a-39a6cd7be49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:50.892455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:50.975487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1901d-af51-4ba1-924b-64598fe6ce6b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:50.996405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 396/781] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:51.011735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6b0d2-432f-4206-831a-39a6cd7be49b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:51.018851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:51.019642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:51.020033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:51.020442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:51.021942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=2cdbdedd9ba34e81916d5104157dfd10) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:51.022365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:51.023902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:51.031393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 386/782] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:50 2022] GET /identity/v3/role_assignments?user.id=2cdbdedd9ba34e81916d5104157dfd10&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:51.074595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:51.127967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6b0d2-432f-4206-831a-39a6cd7be49b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:51.165850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740166b1-501d-4ba0-945f-0050eeaa4c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:51.166666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740166b1-501d-4ba0-945f-0050eeaa4c33 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:51.167149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740166b1-501d-4ba0-945f-0050eeaa4c33 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:51.167490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740166b1-501d-4ba0-945f-0050eeaa4c33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:51.179455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740166b1-501d-4ba0-945f-0050eeaa4c33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-740166b1-501d-4ba0-945f-0050eeaa4c33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:51.188538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:51.189693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:51.190589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 397/783] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:51.196388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:51.331931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef5e5e-04d5-466c-966a-bbed00a54c41 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:51.354188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:51.377897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:51.379543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:51.379543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:51.379543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:51.379893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:51.380235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:51.382191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:51.396525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 387/784] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:51.402078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740166b1-501d-4ba0-945f-0050eeaa4c33 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:51.624853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740166b1-501d-4ba0-945f-0050eeaa4c33 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:51.656864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:51.679785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:51.680971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:51.681464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:51.682033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:51.699602 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:51.701422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:51.723642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:51.725180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:51.726094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:51.726094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:51.726094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:51.726094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:51.728344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 398/785] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:51.737220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:51.888151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be26329b-f401-4c33-b54f-fce646ceee19 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:51.937771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ebecd-68ad-44db-9867-f5487cc5c803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:51.938454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ebecd-68ad-44db-9867-f5487cc5c803 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:51.938741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ebecd-68ad-44db-9867-f5487cc5c803 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:51.939019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ebecd-68ad-44db-9867-f5487cc5c803 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:51.939747 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ebecd-68ad-44db-9867-f5487cc5c803 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:51.940055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ebecd-68ad-44db-9867-f5487cc5c803 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:51.941447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ebecd-68ad-44db-9867-f5487cc5c803 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:51.957283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 388/786] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:51.962568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.024159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcd4a0-44d1-4822-a16b-cf01e2af3245 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.025082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcd4a0-44d1-4822-a16b-cf01e2af3245 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.025082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcd4a0-44d1-4822-a16b-cf01e2af3245 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.026730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 389/787] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:52.042244 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163952f-80f3-4d3e-8e81-4c795956919d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.042541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163952f-80f3-4d3e-8e81-4c795956919d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.042820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163952f-80f3-4d3e-8e81-4c795956919d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.081491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3163952f-80f3-4d3e-8e81-4c795956919d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:52.082859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3163952f-80f3-4d3e-8e81-4c795956919d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:52.105487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2376c5-8e6a-4bc9-97e8-88af9045b957 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.149521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.149707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.150505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.150714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.152185 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.172952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.174429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:52.196741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:52.197946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:52.199293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 399/788] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:52.201827 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 390/789] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:52.218699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.231740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163952f-80f3-4d3e-8e81-4c795956919d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.268872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.353627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163952f-80f3-4d3e-8e81-4c795956919d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.372615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822cfed-24f6-4f5d-b06b-afc9902718ba None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.390444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.391372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.391762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.392051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.392750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.394134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.411867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.412653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.412929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.416136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.416136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:52.416136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.416136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:52.419787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:52.420734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 25 05:41:52.421587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:52.422845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 391/790] 10.222.0.40 () {56 vars in 1129 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:52.429715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.457061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 400/791] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:52.464041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.495839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.612726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.626645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9e858-a2df-48ed-b2ca-359dc0ccfd4c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.639805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab96da1-4a3b-44ca-9036-1325fcec2e9a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.662817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755effda-a691-472b-bbc3-826885f5f337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.663522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755effda-a691-472b-bbc3-826885f5f337 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.663807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755effda-a691-472b-bbc3-826885f5f337 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.664088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755effda-a691-472b-bbc3-826885f5f337 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.664870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755effda-a691-472b-bbc3-826885f5f337 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:52.665184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755effda-a691-472b-bbc3-826885f5f337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.666554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755effda-a691-472b-bbc3-826885f5f337 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.671428 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 392/792] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:52.677145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208a5e-37d4-49e4-9740-753a59cd1448 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.678303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208a5e-37d4-49e4-9740-753a59cd1448 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.678594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea208a5e-37d4-49e4-9740-753a59cd1448 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.680252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 393/793] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:52.681039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208a5e-37d4-49e4-9740-753a59cd1448 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.681759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1cae5-4502-4e18-97ad-8ffdf5ae1a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.682848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1cae5-4502-4e18-97ad-8ffdf5ae1a15 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.683335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1cae5-4502-4e18-97ad-8ffdf5ae1a15 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.684044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1cae5-4502-4e18-97ad-8ffdf5ae1a15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.695873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1cae5-4502-4e18-97ad-8ffdf5ae1a15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.697821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1cae5-4502-4e18-97ad-8ffdf5ae1a15 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-d6b1cae5-4502-4e18-97ad-8ffdf5ae1a15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:52.718322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:52.719220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:52.720551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:52.720551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:52.720551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:52.720551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:52.720551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 401/794] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:52.721382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209a8b-4253-49f2-b255-cdc06d19fa18 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.721498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209a8b-4253-49f2-b255-cdc06d19fa18 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.723005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb209a8b-4253-49f2-b255-cdc06d19fa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.745367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fb209a8b-4253-49f2-b255-cdc06d19fa18 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:52.746556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fb209a8b-4253-49f2-b255-cdc06d19fa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:52.796489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.805015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea208a5e-37d4-49e4-9740-753a59cd1448 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.818535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 402/795] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:52.819715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb209a8b-4253-49f2-b255-cdc06d19fa18 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.859466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.860367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.860713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.861138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.861922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:52.862297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.864378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.876840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 394/796] 10.222.0.40 () {56 vars in 1124 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:52.876840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:52.912317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:52.954723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb209a8b-4253-49f2-b255-cdc06d19fa18 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:52.992855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:52.993855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:52.994320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:52.994772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:52.995674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:52.996203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:52.997795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:52.999949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad0bec-f346-4ed4-b68e-14be1f35961b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.014964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 403/797] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:53.016193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.041843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f597bf-3b67-4d7b-9a17-ab4d6e26b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.042743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f597bf-3b67-4d7b-9a17-ab4d6e26b542 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.043096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f597bf-3b67-4d7b-9a17-ab4d6e26b542 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.043446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f597bf-3b67-4d7b-9a17-ab4d6e26b542 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.054748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f597bf-3b67-4d7b-9a17-ab4d6e26b542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-a5f597bf-3b67-4d7b-9a17-ab4d6e26b542 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:53.062602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:53.063528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:53.064395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:53.065901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 395/798] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:53.066761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f597bf-3b67-4d7b-9a17-ab4d6e26b542 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.094304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:53.133918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04007b5-44f1-4820-8659-82d5a4924d7a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.160082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:53.173383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff162f7-6794-4b1c-9924-23d67b210d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.174553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff162f7-6794-4b1c-9924-23d67b210d63 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.175285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff162f7-6794-4b1c-9924-23d67b210d63 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.176170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff162f7-6794-4b1c-9924-23d67b210d63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.177216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff162f7-6794-4b1c-9924-23d67b210d63 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:53.177920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f597bf-3b67-4d7b-9a17-ab4d6e26b542 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.179293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff162f7-6794-4b1c-9924-23d67b210d63 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:53.215316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.216147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.216502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.216850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.223946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:53.226187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:53.240597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:53.241559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:53.242516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 396/799] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:53.242996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.296039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 404/800] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:53.297066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff162f7-6794-4b1c-9924-23d67b210d63 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.371188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:53.393828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb669e35-fca5-42b8-9c71-24dfcc653b2f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.440741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9b794-ba6d-49bc-9a9e-16710dd3d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.441428 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b794-ba6d-49bc-9a9e-16710dd3d16e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.441716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b794-ba6d-49bc-9a9e-16710dd3d16e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.441999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b794-ba6d-49bc-9a9e-16710dd3d16e None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.443282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9b794-ba6d-49bc-9a9e-16710dd3d16e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:53.443601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9b794-ba6d-49bc-9a9e-16710dd3d16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:53.445060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9b794-ba6d-49bc-9a9e-16710dd3d16e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:53.446609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff162f7-6794-4b1c-9924-23d67b210d63 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.455947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 397/801] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:53.457612 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83d395-a32f-4b19-b5a4-1355e9a9b311 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.457718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83d395-a32f-4b19-b5a4-1355e9a9b311 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.457914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83d395-a32f-4b19-b5a4-1355e9a9b311 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.464045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 398/802] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:53.464928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83d395-a32f-4b19-b5a4-1355e9a9b311 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.492886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e83f2b-5d9c-4547-9439-71734b1c5056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.493793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83f2b-5d9c-4547-9439-71734b1c5056 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.494229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83f2b-5d9c-4547-9439-71734b1c5056 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.494686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e83f2b-5d9c-4547-9439-71734b1c5056 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.495489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e83f2b-5d9c-4547-9439-71734b1c5056 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:53.495805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e83f2b-5d9c-4547-9439-71734b1c5056 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:53.497290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e83f2b-5d9c-4547-9439-71734b1c5056 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:53.512717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 405/803] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:53.514305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807685-6f20-4d85-9fae-4eea03d5078d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.514639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807685-6f20-4d85-9fae-4eea03d5078d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.514927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807685-6f20-4d85-9fae-4eea03d5078d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.533169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4e807685-6f20-4d85-9fae-4eea03d5078d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:53.534380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4e807685-6f20-4d85-9fae-4eea03d5078d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:53.564469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:53.580366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83d395-a32f-4b19-b5a4-1355e9a9b311 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.608203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 406/804] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:53.608203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb3f41-259c-4548-9a01-4908e2976aba None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.608203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb3f41-259c-4548-9a01-4908e2976aba None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.608203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb3f41-259c-4548-9a01-4908e2976aba None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.611993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 407/805] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:53.613020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb3f41-259c-4548-9a01-4908e2976aba None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.633221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bac6f-4e16-4285-8853-9a0338ff79ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.634295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bac6f-4e16-4285-8853-9a0338ff79ba None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.634923 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bac6f-4e16-4285-8853-9a0338ff79ba None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.635205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bac6f-4e16-4285-8853-9a0338ff79ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.642432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:53.646297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bac6f-4e16-4285-8853-9a0338ff79ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:53.647861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bac6f-4e16-4285-8853-9a0338ff79ba None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-506bac6f-4e16-4285-8853-9a0338ff79ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:53.666639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:53.667634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:53.668572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:53.668572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:53.668572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:53.668572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:53.668572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 399/806] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:53.669353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835bdbc-cc72-4826-be2e-b3716cd7b121 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.669442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835bdbc-cc72-4826-be2e-b3716cd7b121 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.669948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835bdbc-cc72-4826-be2e-b3716cd7b121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.687250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b835bdbc-cc72-4826-be2e-b3716cd7b121 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:53.688471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b835bdbc-cc72-4826-be2e-b3716cd7b121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:53.732235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb3f41-259c-4548-9a01-4908e2976aba None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.776215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 400/807] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:53.776215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835bdbc-cc72-4826-be2e-b3716cd7b121 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.779704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5f0ec-32fc-4a19-ad71-439438a65a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.781058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f0ec-32fc-4a19-ad71-439438a65a69 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.781635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f0ec-32fc-4a19-ad71-439438a65a69 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.781918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5f0ec-32fc-4a19-ad71-439438a65a69 None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/cf71e1ef5acb429b96219888830181d3/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.784660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5f0ec-32fc-4a19-ad71-439438a65a69 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=cf71e1ef5acb429b96219888830181d3, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:53.787357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5f0ec-32fc-4a19-ad71-439438a65a69 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:53.848255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 408/808] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:53 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/cf71e1ef5acb429b96219888830181d3/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:53.849167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5f0ec-32fc-4a19-ad71-439438a65a69 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.881435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835bdbc-cc72-4826-be2e-b3716cd7b121 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:53.890108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:53.910594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:53.928952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:53.930308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:53.930871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:53.931441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] PATH_INFO: `/v3/services/network` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:53.932391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:53.934132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:53.949745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:53.950834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 25 05:41:53.951729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:53.954785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 401/809] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:53 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:53.955951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:53.990378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5f0ec-32fc-4a19-ad71-439438a65a69 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.036369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.037921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.038206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.038483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.039906 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:54.040298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:54.041817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:54.055032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 409/810] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:54.057033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.086027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66450c6-080b-481b-82fe-12aee3149aeb None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.132676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bd22f-f8dd-4e8b-80dd-c50019e675a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.134216 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bd22f-f8dd-4e8b-80dd-c50019e675a0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.134689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bd22f-f8dd-4e8b-80dd-c50019e675a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.135539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bd22f-f8dd-4e8b-80dd-c50019e675a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.150631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363bd22f-f8dd-4e8b-80dd-c50019e675a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:54.151374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:54.159408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c231dcee-bb6b-4218-befc-3baa994df0e1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-363bd22f-f8dd-4e8b-80dd-c50019e675a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:54.161023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:54.162005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:54.162831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:54.165548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 402/811] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:54.170519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bd22f-f8dd-4e8b-80dd-c50019e675a0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.201126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.201851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.202141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.202443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.203141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] RBAC: Adding query filter params (name=network) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:54.203450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:54.204899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:54.210663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 410/812] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:54.211555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.233826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:54.302897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bd22f-f8dd-4e8b-80dd-c50019e675a0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.321805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b66603-89d3-4fac-be4a-f69cbc8072ff None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.341252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.342407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.343367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.343873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.354970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:54.356697 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:54.357570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.358328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.358682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.359071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.359947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] RBAC: Adding query filter params (name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:54.360350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:54.361796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:54.376434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 411/813] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:54.376434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:54.380006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:54.381581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:41:54.382565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:54.384416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 403/814] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:41:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:54.385469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.400787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:54.490867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a21e18-5ed3-4f44-a1c5-4adcfa05f05e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.507789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440ba4-7eb2-4997-aba6-771f53963430 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.532867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:54.539680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.541293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.541787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.542237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.543126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] RBAC: Adding query filter params (type=network) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:54.544412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:54.545089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.545531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:54.546193 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.546569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.546918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.553103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 412/815] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:41:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:54.554249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.559941 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:54.561881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:54.581288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:54.582222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:54.583131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:54.583131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:54.583131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:54.583131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:54.587421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 404/816] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:54.588376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.656510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:54.682879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0244347-5ee5-4217-b088-f9edff120274 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.695549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df1ec06-ef33-4094-a19e-c594c9b87fab None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:54.726792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.727724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.728263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.728730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.729748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:54.730244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:54.730350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b42164-1b1c-4d73-877e-f9f506554299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:54.731017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b42164-1b1c-4d73-877e-f9f506554299 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:54.731367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b42164-1b1c-4d73-877e-f9f506554299 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:54.731720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b42164-1b1c-4d73-877e-f9f506554299 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:54.732594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:54.732703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b42164-1b1c-4d73-877e-f9f506554299 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:54.734126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b42164-1b1c-4d73-877e-f9f506554299 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:54.757262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 413/817] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:41:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:54.758356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.820611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 405/818] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:41:54 2022] POST /identity/v3/endpoints => generated 327 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:41:54.821627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b42164-1b1c-4d73-877e-f9f506554299 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:54.856040 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.013993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a63ba-5045-4d6d-bef1-67fbf0b26414 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.069041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.076731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.077407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.078444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.078731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.079479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:55.079790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.081502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.086624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b42164-1b1c-4d73-877e-f9f506554299 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.100210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 414/819] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:55.107539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.133355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb48d9-ea72-4709-875b-2db38bf9d50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.134663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb48d9-ea72-4709-875b-2db38bf9d50d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.135459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb48d9-ea72-4709-875b-2db38bf9d50d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.136429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb48d9-ea72-4709-875b-2db38bf9d50d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.219857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae1ce4-2271-46dd-bbce-431ea3db4d97 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.235918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb48d9-ea72-4709-875b-2db38bf9d50d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.238471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb48d9-ea72-4709-875b-2db38bf9d50d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.252778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.260523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.261467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.268420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.268420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.270158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.272151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:55.290199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:55.291184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:55.292565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 415/820] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:55.297269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.361663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.387624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 406/821] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:55.389335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb48d9-ea72-4709-875b-2db38bf9d50d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.423723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c77a4-ef4d-4d50-a8ac-9fd6c03e625a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.462057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562b686-d850-4bf6-ab74-0a5ae4ec713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.462797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562b686-d850-4bf6-ab74-0a5ae4ec713b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.463125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562b686-d850-4bf6-ab74-0a5ae4ec713b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.463402 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562b686-d850-4bf6-ab74-0a5ae4ec713b None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.464303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562b686-d850-4bf6-ab74-0a5ae4ec713b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:55.464615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562b686-d850-4bf6-ab74-0a5ae4ec713b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.466297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562b686-d850-4bf6-ab74-0a5ae4ec713b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.488895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 416/822] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:55.490398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbcf91-df73-4b70-b446-5dcc481ae6aa None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.490702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbcf91-df73-4b70-b446-5dcc481ae6aa None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.491117 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbcf91-df73-4b70-b446-5dcc481ae6aa None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.492661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 417/823] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:55.493238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbcf91-df73-4b70-b446-5dcc481ae6aa None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.520257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb48d9-ea72-4709-875b-2db38bf9d50d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.530854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.557346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f15eb1-0d2b-41cf-947f-a5469422b89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.558142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15eb1-0d2b-41cf-947f-a5469422b89e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.558425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15eb1-0d2b-41cf-947f-a5469422b89e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.558775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f15eb1-0d2b-41cf-947f-a5469422b89e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.569221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f15eb1-0d2b-41cf-947f-a5469422b89e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.571077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f15eb1-0d2b-41cf-947f-a5469422b89e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-96f15eb1-0d2b-41cf-947f-a5469422b89e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:55.589143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:55.590085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:55.590967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:55.590967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:55.590967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:55.590967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:55.590967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 407/824] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:55.591963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8eae-193b-4ff0-a897-0aaf6ac67804 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.591963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8eae-193b-4ff0-a897-0aaf6ac67804 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.592273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cb8eae-193b-4ff0-a897-0aaf6ac67804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.609567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-50cb8eae-193b-4ff0-a897-0aaf6ac67804 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:55.610725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-50cb8eae-193b-4ff0-a897-0aaf6ac67804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:55.613434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfbcf91-df73-4b70-b446-5dcc481ae6aa None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.624509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.650873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.651715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.652048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.652353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.664562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.666022 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.683122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 408/825] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:55.689046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8eae-193b-4ff0-a897-0aaf6ac67804 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:55.689487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:55.690363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:55.691290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:55.691290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:55.691290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:55.691290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:55.697006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 418/826] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:55.697756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.803686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.814466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cb8eae-193b-4ff0-a897-0aaf6ac67804 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.818438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c95104-059c-498f-8287-107f5da55a55 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.851382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.851715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:55.852058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.852370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.852465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:55.852650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.852747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:55.852933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:55.853374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:55.853769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:55.853872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.854014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:55.855169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.855316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:55.871964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 419/827] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:55.872146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 409/828] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:55.872730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.873415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:55.894494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:55.981529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67258a11-ca2b-45e6-a5f4-2d02f9a14422 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:55.985667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143bffe-0400-478c-81fb-8805ef4b33b3 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.019740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.020832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.021259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.021700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.022496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.023057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80fe0e-74e0-4309-8e3a-82fcffa154e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.023858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80fe0e-74e0-4309-8e3a-82fcffa154e9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.024171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80fe0e-74e0-4309-8e3a-82fcffa154e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.024455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80fe0e-74e0-4309-8e3a-82fcffa154e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.024774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.031805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80fe0e-74e0-4309-8e3a-82fcffa154e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.033271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80fe0e-74e0-4309-8e3a-82fcffa154e9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:56.036840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:56.037997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:56.038851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 420/829] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:56.039637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-4f80fe0e-74e0-4309-8e3a-82fcffa154e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:56.047972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:56.049151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:56.050382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 410/830] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:56.051077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91635ea-a395-474b-87a5-50e98c3d4ea1 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.051168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91635ea-a395-474b-87a5-50e98c3d4ea1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.051385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91635ea-a395-474b-87a5-50e98c3d4ea1 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.053436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 411/831] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:56.054348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91635ea-a395-474b-87a5-50e98c3d4ea1 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.066403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:56.133109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:56.148882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cde513-dc73-499f-a5ff-1073e24c24eb None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.155547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91635ea-a395-474b-87a5-50e98c3d4ea1 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.184658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f01a4-14f0-41e1-8c95-f00efea101ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.185448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f01a4-14f0-41e1-8c95-f00efea101ee None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.185730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f01a4-14f0-41e1-8c95-f00efea101ee None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.186005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f01a4-14f0-41e1-8c95-f00efea101ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.188994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.189649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.189934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.190208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.190910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:56.191243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.192702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.192817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f01a4-14f0-41e1-8c95-f00efea101ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.194128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f01a4-14f0-41e1-8c95-f00efea101ee None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.197603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 412/832] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:56.198373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-8c2f01a4-14f0-41e1-8c95-f00efea101ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:56.208719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:56.209718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:56.210626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 421/833] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:56.211662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564cc94-2cec-4d9f-9cb5-9f578a408393 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.211949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564cc94-2cec-4d9f-9cb5-9f578a408393 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.212256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564cc94-2cec-4d9f-9cb5-9f578a408393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.229569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2564cc94-2cec-4d9f-9cb5-9f578a408393 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:56.230757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2564cc94-2cec-4d9f-9cb5-9f578a408393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:56.278467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:56.296859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 422/834] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:56.297726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564cc94-2cec-4d9f-9cb5-9f578a408393 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.302809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c303ef5-fa16-4235-b888-51c783a88d48 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.337397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.338060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.338343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.338621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.339406 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:56.339714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.341171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.352940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 413/835] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:56.353733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.373193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:56.397916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564cc94-2cec-4d9f-9cb5-9f578a408393 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.431511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.432216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.432503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.432782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.433491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:56.434058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.435168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.439656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 423/836] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:56.440380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.455539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d136e-3f0c-4aeb-9217-7920fed79d65 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.497591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.498396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.498683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.498964 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.499773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:56.500092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.501607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.503657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:56.517063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 414/837] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:56.518137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.562007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1876a8d-f3cb-4dab-9843-82da39a86e18 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.607264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ff5c7-a3c2-4d61-9d1a-5566ea9a6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.608215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff5c7-a3c2-4d61-9d1a-5566ea9a6a1b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.608501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff5c7-a3c2-4d61-9d1a-5566ea9a6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.608781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ff5c7-a3c2-4d61-9d1a-5566ea9a6a1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.617667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:56.621592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ff5c7-a3c2-4d61-9d1a-5566ea9a6a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-665ff5c7-a3c2-4d61-9d1a-5566ea9a6a1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:56.629367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:56.630866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 424/838] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:56.632704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ff5c7-a3c2-4d61-9d1a-5566ea9a6a1b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.644457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83683-ada1-44eb-a337-e43f8d869625 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.683464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1377c-93c4-4155-ba92-e57bc0dde227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.684437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1377c-93c4-4155-ba92-e57bc0dde227 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.684778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1377c-93c4-4155-ba92-e57bc0dde227 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.685054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1377c-93c4-4155-ba92-e57bc0dde227 None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/2cdbdedd9ba34e81916d5104157dfd10/roles/2c3d129fa9da4379acd24c88e17b7701` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.701354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1377c-93c4-4155-ba92-e57bc0dde227 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=2cdbdedd9ba34e81916d5104157dfd10, role_id=2c3d129fa9da4379acd24c88e17b7701)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.702793 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde1377c-93c4-4155-ba92-e57bc0dde227 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.735628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ff5c7-a3c2-4d61-9d1a-5566ea9a6a1b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.770513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.771536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.771858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.772158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.772994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=e6a0f84bf3f245bdab1391a2451cb4c0) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:56.773354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.774829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.784363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 415/839] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:41:56 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/2cdbdedd9ba34e81916d5104157dfd10/roles/2c3d129fa9da4379acd24c88e17b7701 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:41:56.784363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1377c-93c4-4155-ba92-e57bc0dde227 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:56.791239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 425/840] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/role_assignments?user.id=e6a0f84bf3f245bdab1391a2451cb4c0&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:56.842174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:56.915649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde1377c-93c4-4155-ba92-e57bc0dde227 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:56.957132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:56.957752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:56.958036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:56.958912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:56.960159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:56.960477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:56.962033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:56.972391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 416/841] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:56.978969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:57.052930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:57.096070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90c0c4-4ebd-48c4-a71d-1d53c11f474f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:57.131175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:57.132305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:57.132776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:57.133234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:57.144199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:57.146432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:57.165266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:57.166285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:57.167193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:57.167193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:57.167193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:57.167193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:57.169237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 426/842] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:57.176160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:57.284702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ee1e9-01a1-43e5-97ac-220ae078d0ae None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:57.292083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:57.320656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:57.321330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:57.321614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:57.321891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:57.322618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:57.322928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:57.324318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:57.336743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 417/843] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:57.342013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:57.448199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ed8c5-e546-475b-be82-2f44fa2caa5c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:57.462537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:57.487850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:57.489011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:57.490030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:57.490502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:57.498824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:57.500531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:41:57.517292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:57.518258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 25 05:41:57.519212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:57.522258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 427/844] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:41:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:57.528191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:57.640169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e4630-4f3c-4609-88c1-8c7cb40c4334 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:57.675775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:57.676489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:57.676776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:57.677571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:57.678410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:57.678715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:57.680168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:57.699124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 418/845] 10.222.0.40 () {56 vars in 1135 bytes} [Mon Jul 25 05:41:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:57.704451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:57.824256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184fcdb-43d2-4931-86e6-73ddb3e0c377 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:57.861107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679611c-9908-4aa1-b101-db69dbec8b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:57.862055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679611c-9908-4aa1-b101-db69dbec8b88 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:57.862578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679611c-9908-4aa1-b101-db69dbec8b88 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:57.863059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679611c-9908-4aa1-b101-db69dbec8b88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:57.908191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:57.935060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679611c-9908-4aa1-b101-db69dbec8b88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:57.936853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679611c-9908-4aa1-b101-db69dbec8b88 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:58.041813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 428/846] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:41:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:41:58.047560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:58.074093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:58.154529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cd0bd-97d4-4331-bd6d-06da1a1d58d9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:58.189984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:58.191468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.191901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.192352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.203393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:58.205028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:58.225640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:58.226734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:58.227835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:58.227835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:58.227835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:58.227835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:58.229699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 419/847] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:58.237284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679611c-9908-4aa1-b101-db69dbec8b88 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:58.310866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:58.337568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679611c-9908-4aa1-b101-db69dbec8b88 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:58.373380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:58.374299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.374724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.375152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.376014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:58.376487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:58.378068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:58.389942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 429/848] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:58.396080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:58.480305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:58.507389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c0b5e-33c5-4b69-b6d9-e9d3a18f5507 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:58.543670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa92bb4-0b2f-42f1-b024-7b273e6d396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:58.544515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa92bb4-0b2f-42f1-b024-7b273e6d396b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.544800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa92bb4-0b2f-42f1-b024-7b273e6d396b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.545076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa92bb4-0b2f-42f1-b024-7b273e6d396b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.552723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa92bb4-0b2f-42f1-b024-7b273e6d396b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:58.554175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa92bb4-0b2f-42f1-b024-7b273e6d396b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-9fa92bb4-0b2f-42f1-b024-7b273e6d396b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:41:58.568417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:58.569310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:58.570458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 420/849] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:58.577173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:58.591503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53261059-22e1-4094-8cdd-b81b9a46a8ad None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.591815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53261059-22e1-4094-8cdd-b81b9a46a8ad None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.592094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53261059-22e1-4094-8cdd-b81b9a46a8ad None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.593737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 421/850] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:58.599373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc5672-17f3-4cd8-be4e-498db037cfeb None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.599719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc5672-17f3-4cd8-be4e-498db037cfeb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.599992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc5672-17f3-4cd8-be4e-498db037cfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.617455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-72cc5672-17f3-4cd8-be4e-498db037cfeb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:58.618642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-72cc5672-17f3-4cd8-be4e-498db037cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:58.696680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba199d5-1760-4788-ae8c-3adb3145aae1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:58.698570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 422/851] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:41:58.717310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc5672-17f3-4cd8-be4e-498db037cfeb None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:58.737259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:58.737964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.738253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.738532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.739404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:58.740257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:58.741710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:58.743646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:58.763674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 430/852] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:58.769986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:58.799180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:58.839191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc5672-17f3-4cd8-be4e-498db037cfeb None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:58.885802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30134a33-7be6-4478-b5c6-74608b1eff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:58.886535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30134a33-7be6-4478-b5c6-74608b1eff06 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.886826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30134a33-7be6-4478-b5c6-74608b1eff06 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.887106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30134a33-7be6-4478-b5c6-74608b1eff06 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.888092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30134a33-7be6-4478-b5c6-74608b1eff06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:58.890186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30134a33-7be6-4478-b5c6-74608b1eff06 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:58.894963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f5fde-a292-43b7-8e54-bcd3377988a3 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:58.931365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fef758-8159-4e2b-8374-cfbced277851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:58.932066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fef758-8159-4e2b-8374-cfbced277851 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.932383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fef758-8159-4e2b-8374-cfbced277851 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.932664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fef758-8159-4e2b-8374-cfbced277851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.933508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fef758-8159-4e2b-8374-cfbced277851 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=cf71e1ef5acb429b96219888830181d3) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:58.933824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fef758-8159-4e2b-8374-cfbced277851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:58.935265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fef758-8159-4e2b-8374-cfbced277851 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:58.941037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 431/853] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:41:58 2022] GET /identity/v3/role_assignments?user.id=cf71e1ef5acb429b96219888830181d3&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:58.942515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368cbd8-54bc-48c6-9a7e-5cc438809d90 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.942614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368cbd8-54bc-48c6-9a7e-5cc438809d90 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.942955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368cbd8-54bc-48c6-9a7e-5cc438809d90 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.944454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 432/854] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:41:58.950489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb47d43-ce3b-4b81-8236-1856c5f905f5 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:58.950776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb47d43-ce3b-4b81-8236-1856c5f905f5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:58.951055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb47d43-ce3b-4b81-8236-1856c5f905f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:58.970119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-abb47d43-ce3b-4b81-8236-1856c5f905f5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:58.971350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-abb47d43-ce3b-4b81-8236-1856c5f905f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:58.983245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 423/855] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:41:58 2022] POST /identity/v3/services => generated 251 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:41:59.041402 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:59.072326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 433/856] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:59.093808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30134a33-7be6-4478-b5c6-74608b1eff06 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:59.146172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e17f2-fcc3-49f3-8c06-99d805abb837 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.146499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e17f2-fcc3-49f3-8c06-99d805abb837 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.146740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e17f2-fcc3-49f3-8c06-99d805abb837 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.148327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 434/857] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:41:59.154630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62037b2e-27da-4247-ac19-e9745ae88ac4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.154942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62037b2e-27da-4247-ac19-e9745ae88ac4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.155192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62037b2e-27da-4247-ac19-e9745ae88ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.175921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-62037b2e-27da-4247-ac19-e9745ae88ac4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:41:59.177212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-62037b2e-27da-4247-ac19-e9745ae88ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:41:59.228905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30134a33-7be6-4478-b5c6-74608b1eff06 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:59.258926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 435/858] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:41:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:41:59.268495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:59.273624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dc577-f86e-4300-8f9b-a2a78c2f8624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:59.274429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dc577-f86e-4300-8f9b-a2a78c2f8624 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.274713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dc577-f86e-4300-8f9b-a2a78c2f8624 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.274988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dc577-f86e-4300-8f9b-a2a78c2f8624 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.280084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62037b2e-27da-4247-ac19-e9745ae88ac4 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:59.290655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dc577-f86e-4300-8f9b-a2a78c2f8624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-964dc577-f86e-4300-8f9b-a2a78c2f8624 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:41:59.299631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:59.300558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:41:59.301400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:59.305226 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 424/859] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:59.311061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dc577-f86e-4300-8f9b-a2a78c2f8624 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:59.403231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62037b2e-27da-4247-ac19-e9745ae88ac4 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:59.418519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dc577-f86e-4300-8f9b-a2a78c2f8624 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:59.419409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:59.442045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:59.443850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.444371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.444978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.446244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:59.448273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:59.455814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:59.456538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.456823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.457101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.457901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:59.459001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:59.460376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:59.465083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:59.466006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:41:59.466857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:41:59.472791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 425/860] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:59.472942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 436/861] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Jul 25 05:41:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:41:59.477297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:59.478572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:59.534155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:59.591946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e7fa47-15a0-46cc-91cf-143ea029408a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:59.602916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983a106-4ed5-43bd-8cb0-3977cb0149cf None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:59.629172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:59.629847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.630832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.631118 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.631884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:59.632140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:59.633902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:59.643893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 437/862] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:59.644945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:59.646716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:59.647974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.649277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.649765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.663489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:59.665498 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:41:59.691266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:41:59.692246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:41:59.693770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:41:59.693770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:41:59.693770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:41:59.693770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:41:59.693998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:59.701028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 426/863] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:41:59.707207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:41:59.764136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ff6ca-9350-447c-a244-b7586c232c06 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:59.804784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:59.805553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.805943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.806290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.807069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:59.807441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:59.808928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:59.819191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 438/864] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:41:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:59.842611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:41:59.846430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1515b41-2222-4f62-b2d8-fa50487bd1fc None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:41:59.885546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:41:59.886635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:41:59.887121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:41:59.887583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:41:59.888546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:41:59.889029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:41:59.890617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:41:59.908844 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 427/865] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:41:59.911806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.020418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee74854-0d3f-4083-9a16-30ad3a2ff810 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.029204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:00.056100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.057000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.057288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.057566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.065812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:00.068298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:42:00.083873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:00.084883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:42:00.085814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:00.088237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 439/866] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:41:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:00.091349 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.200465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f5d16-8883-4d05-bf50-7f31c1eb3cea None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.236555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.237336 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.237533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.237817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.238663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:00.238980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:00.240604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:00.263784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 428/867] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:00.265181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.372526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa369ce4-d7ed-4369-a130-4ab79bc8ef0b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.406890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb5473-3867-41c1-80b4-ee321e39d146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.407545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb5473-3867-41c1-80b4-ee321e39d146 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.407830 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb5473-3867-41c1-80b4-ee321e39d146 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.408122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb5473-3867-41c1-80b4-ee321e39d146 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.442345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:00.477449 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb5473-3867-41c1-80b4-ee321e39d146 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:00.478835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb5473-3867-41c1-80b4-ee321e39d146 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:00.586525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 440/868] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:00.594581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.612684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:00.698867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894bce-3241-499b-b701-b1f59516e083 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.734520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.735259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.735545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.735824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.746960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:00.748442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:00.767791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:00.768752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:00.769633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:00.769633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:00.769633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:00.769633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:00.773876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 429/869] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:00.775239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb5473-3867-41c1-80b4-ee321e39d146 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:00.844890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:00.881786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb5473-3867-41c1-80b4-ee321e39d146 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:00.917928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:00.918605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:00.918908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:00.919181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:00.919912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:00.920304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:00.921710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:00.939149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 441/870] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:00.940547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.013338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.054996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba036d08-2326-4b29-823f-cec494ca03ce None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.090281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00944c0a-9cd6-4259-b889-54cb1515c3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.091092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00944c0a-9cd6-4259-b889-54cb1515c3f2 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.091375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00944c0a-9cd6-4259-b889-54cb1515c3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.091651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00944c0a-9cd6-4259-b889-54cb1515c3f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.099851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00944c0a-9cd6-4259-b889-54cb1515c3f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.101527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00944c0a-9cd6-4259-b889-54cb1515c3f2 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-00944c0a-9cd6-4259-b889-54cb1515c3f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:42:01.120360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:01.121329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:01.122236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:42:01.122236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:42:01.122236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:01.122236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:01.122236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:01.122236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:01.122236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:01.122236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:01.124782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 430/871] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:01.131962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.133736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9e27-a53c-4c86-917c-cd6700b2f867 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.134238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9e27-a53c-4c86-917c-cd6700b2f867 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.134761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9e27-a53c-4c86-917c-cd6700b2f867 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.136586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 431/872] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:01.141935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a8cdc-9d8e-49d2-9d9c-295c6c458322 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.142477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a8cdc-9d8e-49d2-9d9c-295c6c458322 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.142967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a8cdc-9d8e-49d2-9d9c-295c6c458322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.160631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-215a8cdc-9d8e-49d2-9d9c-295c6c458322 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:01.162794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-215a8cdc-9d8e-49d2-9d9c-295c6c458322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:01.234237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 432/873] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:01.236316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e433-9a35-43e8-adc6-357551598c05 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.236316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e433-9a35-43e8-adc6-357551598c05 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.236316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693e433-9a35-43e8-adc6-357551598c05 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.240177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 433/874] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:01.242968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6a3c-568c-4ad6-b714-f848820f97ac None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.243258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6a3c-568c-4ad6-b714-f848820f97ac None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.243540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e6a3c-568c-4ad6-b714-f848820f97ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.256043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3fb70-f191-4a43-96f2-fa5f9e16eb37 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.260568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ea6e6a3c-568c-4ad6-b714-f848820f97ac None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:01.261761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ea6e6a3c-568c-4ad6-b714-f848820f97ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:01.285125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.293966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.294630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.294918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.295192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.295980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:01.296322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.297754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.313720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 442/875] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:01.313720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.328908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.340666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 434/876] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:01.341413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e6a3c-568c-4ad6-b714-f848820f97ac None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.431817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54698776-bef3-4b3d-b4bc-4dc1cc0dc899 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.445431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e6a3c-568c-4ad6-b714-f848820f97ac None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.470671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.471478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.471763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.472043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.472730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.474671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.480627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.481287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.481574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.481853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.482686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=2cdbdedd9ba34e81916d5104157dfd10) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:01.483027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.484442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:01.486698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:01.487667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:01.488564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 443/877] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:01.489421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.500859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 435/878] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/role_assignments?user.id=2cdbdedd9ba34e81916d5104157dfd10&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:01.500859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.561347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.574448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.627912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a0b27-520f-462c-8d28-14b17c832503 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.634255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a522d77f-0bf6-4c19-8677-fcab3681617b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.669823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.670675 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.671006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.671400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.672140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:01.672462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.673980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.682870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 436/879] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:01.687653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.689778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.690624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.690910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.691188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.691854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.694731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:01.707532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:01.709192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:42:01.710056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:01.710787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 444/880] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:01.715800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:01.818042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5cc03-cad5-4d68-bebb-c05a92e6dd54 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.842839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.860927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e17af-c352-499c-919a-bf5693e68d7a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:01.862783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.863563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.863859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.864718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.866070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:01.866385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.867777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.873290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 437/881] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:01.881174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:01.901447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ce338-3d5a-4527-a73e-077582c55856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:01.902148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ce338-3d5a-4527-a73e-077582c55856 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:01.902435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ce338-3d5a-4527-a73e-077582c55856 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:01.902715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ce338-3d5a-4527-a73e-077582c55856 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:01.903486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ce338-3d5a-4527-a73e-077582c55856 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:01.903871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ce338-3d5a-4527-a73e-077582c55856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:01.905412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ce338-3d5a-4527-a73e-077582c55856 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:01.911667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 445/882] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:01.917067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:02.039993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343249ec-0b5b-4afd-a6da-8c305cb7e51e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:02.066808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774082f-6f6c-4f22-ab40-a8221e115a7f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:02.067201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774082f-6f6c-4f22-ab40-a8221e115a7f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:02.067635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774082f-6f6c-4f22-ab40-a8221e115a7f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:02.069451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 446/883] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:02.075217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc007c8-486d-4e69-8620-b13c02d14ef9 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:02.075578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc007c8-486d-4e69-8620-b13c02d14ef9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:02.075688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:02.075957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc007c8-486d-4e69-8620-b13c02d14ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:02.076364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:02.076656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:02.076942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:02.077666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:02.078073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:02.079553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:02.085800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 438/884] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:42:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:02.091462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:02.094866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-8cc007c8-486d-4e69-8620-b13c02d14ef9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:02.096967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8cc007c8-486d-4e69-8620-b13c02d14ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:02.152287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:02.184731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 447/885] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:02.207147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc007c8-486d-4e69-8620-b13c02d14ef9 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:02.219544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55679fbf-a817-4d94-9ad3-1f8e30c6b16b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:02.266140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:02.266968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:02.267568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:02.267858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:02.268953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] RBAC: Adding query filter params (interface=public, service_id=71da55337c9e49a0bbeff0355c7e93c4, region_id=RegionOne) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:02.269274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:02.271438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:02.284207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 439/886] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 25 05:42:02 2022] GET /identity/v3/endpoints?service_id=71da55337c9e49a0bbeff0355c7e93c4&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:02.287320 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:02.315327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc007c8-486d-4e69-8620-b13c02d14ef9 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:02.326352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:02.361106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b90e02-5b72-47de-b4d8-debd08366218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:02.363200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b90e02-5b72-47de-b4d8-debd08366218 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:02.364212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b90e02-5b72-47de-b4d8-debd08366218 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:02.365377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b90e02-5b72-47de-b4d8-debd08366218 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:02.366622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b90e02-5b72-47de-b4d8-debd08366218 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:02.368964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b90e02-5b72-47de-b4d8-debd08366218 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:02.411478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e263fff-5569-487e-920a-92601e5449fa None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:02.423069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:02.447385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120956d-bb16-4a59-ad3c-9ea567306528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:02.448072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120956d-bb16-4a59-ad3c-9ea567306528 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:02.448394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120956d-bb16-4a59-ad3c-9ea567306528 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:02.448678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120956d-bb16-4a59-ad3c-9ea567306528 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:02.449362 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120956d-bb16-4a59-ad3c-9ea567306528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:02.450886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120956d-bb16-4a59-ad3c-9ea567306528 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:02.464154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 440/887] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:42:02 2022] GET /identity/v3/services => generated 1011 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:42:02.480034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 448/888] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:42:02 2022] POST /identity/v3/services => generated 269 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:03.724605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d0ad-4b03-4738-81d6-21f7be57baf8 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.725163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d0ad-4b03-4738-81d6-21f7be57baf8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.725163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74d0ad-4b03-4738-81d6-21f7be57baf8 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.726821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 441/889] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:03.732140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397bb3-d47d-482c-947e-afe453451641 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.732548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397bb3-d47d-482c-947e-afe453451641 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.732740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c397bb3-d47d-482c-947e-afe453451641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.750461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-7c397bb3-d47d-482c-947e-afe453451641 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:03.751670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7c397bb3-d47d-482c-947e-afe453451641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:03.829979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 449/890] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:03.854057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74d0ad-4b03-4738-81d6-21f7be57baf8 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:03.961893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74d0ad-4b03-4738-81d6-21f7be57baf8 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:03.986424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ccd2-f436-4f76-a022-8272bfc20159 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.987036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ccd2-f436-4f76-a022-8272bfc20159 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.987546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d4ccd2-f436-4f76-a022-8272bfc20159 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.993164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 450/891] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:03.994973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e9c7c-1bc5-41c8-9507-4318233bbb8b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.995473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e9c7c-1bc5-41c8-9507-4318233bbb8b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.995958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e9c7c-1bc5-41c8-9507-4318233bbb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:03.997299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de87f6-8a9d-4a37-abbe-dd0413125f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:03.998138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de87f6-8a9d-4a37-abbe-dd0413125f6d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:03.998405 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de87f6-8a9d-4a37-abbe-dd0413125f6d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:03.998681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de87f6-8a9d-4a37-abbe-dd0413125f6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.010141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de87f6-8a9d-4a37-abbe-dd0413125f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:04.014768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-855e9c7c-1bc5-41c8-9507-4318233bbb8b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:04.016239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-855e9c7c-1bc5-41c8-9507-4318233bbb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-34de87f6-8a9d-4a37-abbe-dd0413125f6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:04.018387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:04.019325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:04.020895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 442/892] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 25 05:42:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:04.029612 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de87f6-8a9d-4a37-abbe-dd0413125f6d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.077219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:04.112915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 451/893] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:04.137809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e9c7c-1bc5-41c8-9507-4318233bbb8b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.166112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de87f6-8a9d-4a37-abbe-dd0413125f6d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.167842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:04.210603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229daab-7565-4144-a689-77590a271703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.211442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229daab-7565-4144-a689-77590a271703 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.211889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229daab-7565-4144-a689-77590a271703 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.212195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229daab-7565-4144-a689-77590a271703 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.213199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229daab-7565-4144-a689-77590a271703 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:04.213311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229daab-7565-4144-a689-77590a271703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.214800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229daab-7565-4144-a689-77590a271703 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.228287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 443/894] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 25 05:42:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:04.229960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229daab-7565-4144-a689-77590a271703 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.256510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e9c7c-1bc5-41c8-9507-4318233bbb8b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.298016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf64ef-cb27-4543-86c4-7b7a6e473412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.299023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf64ef-cb27-4543-86c4-7b7a6e473412 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.299499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf64ef-cb27-4543-86c4-7b7a6e473412 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.300057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf64ef-cb27-4543-86c4-7b7a6e473412 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.301025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf64ef-cb27-4543-86c4-7b7a6e473412 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.303159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf64ef-cb27-4543-86c4-7b7a6e473412 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.332994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:04.339173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229daab-7565-4144-a689-77590a271703 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.376770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 452/895] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:42:04 2022] POST /identity/v3/services => generated 241 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:42:04.383222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599593c-28ff-44cf-bf5a-ce5b85e34e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.384041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599593c-28ff-44cf-bf5a-ce5b85e34e34 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.384359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599593c-28ff-44cf-bf5a-ce5b85e34e34 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.384693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599593c-28ff-44cf-bf5a-ce5b85e34e34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.397576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599593c-28ff-44cf-bf5a-ce5b85e34e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.399123 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f599593c-28ff-44cf-bf5a-ce5b85e34e34 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-f599593c-28ff-44cf-bf5a-ce5b85e34e34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:04.421006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:04.421973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:04.422862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:04.422862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:04.422862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:04.422862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:04.422862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 444/896] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:04.433348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf64ef-cb27-4543-86c4-7b7a6e473412 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.530362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:04.549683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf64ef-cb27-4543-86c4-7b7a6e473412 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.564700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b4fc-704d-455c-a057-2e37e93ccc05 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.564997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b4fc-704d-455c-a057-2e37e93ccc05 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.565279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a93b4fc-704d-455c-a057-2e37e93ccc05 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.568229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 445/897] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:42:04.573435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb7c43-6fa9-45cf-90e4-59a6dccfffc9 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.575680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb7c43-6fa9-45cf-90e4-59a6dccfffc9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.575680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb7c43-6fa9-45cf-90e4-59a6dccfffc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.591735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.592436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.592718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.592989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.593712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:04.594015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.594397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-53fb7c43-6fa9-45cf-90e4-59a6dccfffc9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:04.595435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.595570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-53fb7c43-6fa9-45cf-90e4-59a6dccfffc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:04.610338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 453/898] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:04.616089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.680159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 446/899] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:04.681487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f2959-f765-437c-8a89-387dec5fc981 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.681781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f2959-f765-437c-8a89-387dec5fc981 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.682063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f2959-f765-437c-8a89-387dec5fc981 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.683465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 447/900] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:04.688661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45cc9a-ef1a-4282-a495-b12ab43f783a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.688952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45cc9a-ef1a-4282-a495-b12ab43f783a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.689234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45cc9a-ef1a-4282-a495-b12ab43f783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.708521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-fe45cc9a-ef1a-4282-a495-b12ab43f783a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:04.709764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fe45cc9a-ef1a-4282-a495-b12ab43f783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:04.731424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a3eb3-4cf3-4e93-b7b3-bec38134c261 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.773665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.774495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.774782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.775055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.792575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 448/901] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:04.792575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45cc9a-ef1a-4282-a495-b12ab43f783a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.793137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.793227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:04.793299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:42:04.811733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:04.812982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:42:04.813986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:04.816276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 454/902] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:42:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:04.817431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.822871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:04.894438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45cc9a-ef1a-4282-a495-b12ab43f783a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.921667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720b6908-59f2-4148-9dc7-b7dac653a2a6 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:04.931625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.932469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.932760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.933068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.935367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.935367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:04.947730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:04.949043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:04.949869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 449/903] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:42:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:04.950684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:04.966666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:04.967487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:04.967766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:04.968038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:04.968728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:04.970099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:04.983440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:04.984468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:04.985354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 455/904] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Jul 25 05:42:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:04.987091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.028192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:05.056795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:05.065733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37609dc1-8b5f-439f-875c-13510f91d4be None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.093849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50fa0-d2ba-4086-b9bb-4e495b96efde None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.105093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.105782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.106068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.106383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.107227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:05.107542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.109007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.126870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 450/905] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 25 05:42:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:05.127694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.135620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.136566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.137031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.137515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.138226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:05.138534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.139894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.150125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 456/906] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:42:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:05.150125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.238938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500650c1-d9a0-49e5-bef4-00cd61eb1efb None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.257409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e42f4-63ee-4f52-8f5e-649ef2ad1329 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.266869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:05.276631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.277300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.278015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.278015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.278535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:05.278839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.280256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.288144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 451/907] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:05.288144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.295186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:05.300506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506228c6-6415-4fd9-aaa1-2adf5888a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.301161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506228c6-6415-4fd9-aaa1-2adf5888a38a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.301474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506228c6-6415-4fd9-aaa1-2adf5888a38a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.301759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506228c6-6415-4fd9-aaa1-2adf5888a38a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.372042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506228c6-6415-4fd9-aaa1-2adf5888a38a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.373484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506228c6-6415-4fd9-aaa1-2adf5888a38a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.401666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfeeb29-105c-43cc-9cc1-5a49c06f3638 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.436534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.437208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.437490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.437768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.438502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:05.438813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.440204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.450789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 452/908] 10.222.0.40 () {56 vars in 1114 bytes} [Mon Jul 25 05:42:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:05.450789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.487648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 457/909] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:05 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:05.488789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506228c6-6415-4fd9-aaa1-2adf5888a38a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.509198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:05.522644 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:05.558676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e1d2-6c42-462f-9bd2-ad9000181580 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.596503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.597223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.597458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.597731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.599002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:05.599002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.600261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.602378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506228c6-6415-4fd9-aaa1-2adf5888a38a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.605431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 453/910] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:05.606421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.641396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f869ede-0e83-4808-8e81-0e36e18103a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.641988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869ede-0e83-4808-8e81-0e36e18103a5 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.642291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869ede-0e83-4808-8e81-0e36e18103a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.642553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f869ede-0e83-4808-8e81-0e36e18103a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.643296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f869ede-0e83-4808-8e81-0e36e18103a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.644714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f869ede-0e83-4808-8e81-0e36e18103a5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.714551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf2e10-931a-4a94-8dc1-3174294a4725 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.722031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 458/911] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:42:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:05.722706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f869ede-0e83-4808-8e81-0e36e18103a5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.743776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:05.784273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.785631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.786132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.786589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.799685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.801290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:05.821768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:05.823104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:05.823980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:05.823980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:05.823980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:05.823980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:05.823980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 454/912] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:05.829167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.883109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:05.906668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f869ede-0e83-4808-8e81-0e36e18103a5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:05.956766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:05.957440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:05.958254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:05.958535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:05.959453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] RBAC: Adding query filter params (interface=public, service_id=09051e09a5e04500b3d1ecc92f909acf, region_id=RegionOne) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:05.959772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:05.961307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:05.971446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 459/913] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 25 05:42:05 2022] GET /identity/v3/endpoints?service_id=09051e09a5e04500b3d1ecc92f909acf&interface=public®ion_id=RegionOne => generated 194 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:05.972715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:05.980671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81301529-fa3a-48d5-9d7a-7855a79a1798 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.032497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.033832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.034304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.034989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.036848 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:06.037351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.039887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.057044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 455/914] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:06.062645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.092913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:06.096424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df99bc-cdfe-4f5a-a90a-d47ac261786b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.136463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c60fd-11a8-4c1f-a6a7-709e49c65c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.137153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c60fd-11a8-4c1f-a6a7-709e49c65c93 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.137482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c60fd-11a8-4c1f-a6a7-709e49c65c93 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.137766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c60fd-11a8-4c1f-a6a7-709e49c65c93 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.138464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c60fd-11a8-4c1f-a6a7-709e49c65c93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.140003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c60fd-11a8-4c1f-a6a7-709e49c65c93 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.156374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 460/915] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:42:05 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:42:06.184045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac153-6317-49a8-b3b5-c798e399dd5c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.225988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.227182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.227698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.228213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.236570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.238438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:42:06.254037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:06.255654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:06.256614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:42:06.256614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:42:06.256614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:06.256614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:06.256614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:06.256614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:06.256614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:06.256614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:06.259224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 456/916] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:06.265290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c60fd-11a8-4c1f-a6a7-709e49c65c93 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.286778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:06.376659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c60fd-11a8-4c1f-a6a7-709e49c65c93 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.412670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc526d-9ba8-434f-a065-62c0902734b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.413691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc526d-9ba8-434f-a065-62c0902734b2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.414183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc526d-9ba8-434f-a065-62c0902734b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.414691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc526d-9ba8-434f-a065-62c0902734b2 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.415693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc526d-9ba8-434f-a065-62c0902734b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:06.416239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc526d-9ba8-434f-a065-62c0902734b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.417977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc526d-9ba8-434f-a065-62c0902734b2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.431763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 461/917] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:06.438141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.531379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:06.536804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dfcca-6ddb-4026-acc7-03e39c1b5fd3 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.537093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dfcca-6ddb-4026-acc7-03e39c1b5fd3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.537374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00dfcca-6ddb-4026-acc7-03e39c1b5fd3 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.539020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 462/918] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:06.544870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245796f-ed89-4e6d-889f-64a2dc155ad9 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.545166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245796f-ed89-4e6d-889f-64a2dc155ad9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.545451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245796f-ed89-4e6d-889f-64a2dc155ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.550355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a55fb-0275-43b8-a551-2c1fac23d77d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.567704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-6245796f-ed89-4e6d-889f-64a2dc155ad9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:06.569256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6245796f-ed89-4e6d-889f-64a2dc155ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:06.588301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.589272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.589761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.590254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.591267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] RBAC: Adding query filter params (role.id=1c5a669f88fd44399a230e4f91a6c965, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=2cdbdedd9ba34e81916d5104157dfd10) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:06.591764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.593393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.599666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 457/919] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/role_assignments?user.id=2cdbdedd9ba34e81916d5104157dfd10&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=1c5a669f88fd44399a230e4f91a6c965 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:06.649876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 463/920] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:06.673772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.793479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642149db-dd1d-4dd9-9dfd-ae289898d169 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.829491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.830308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:06.830589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:06.830862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:06.831510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:06.832935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:06.845259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:06.846261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:06.847107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 458/921] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:06.853512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245796f-ed89-4e6d-889f-64a2dc155ad9 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:06.862441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:06.962332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245796f-ed89-4e6d-889f-64a2dc155ad9 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:06.999105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:06.999783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.000070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.000383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.001102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:07.001413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:07.002793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:07.008357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 464/922] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:07.013656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:07.030359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:07.124535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ede238-1001-4b83-8e51-9f830ea7bb7a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:07.158947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:07.159620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.159908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.160288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.161666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:07.161979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:07.163363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:07.168803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 459/923] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:07.174681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:07.252997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:07.284687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b313bcf-ea10-42ef-b50d-8c5f8d950b93 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:07.320898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712da33-6c2f-41f9-82a1-0bddaaa6df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:07.321964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712da33-6c2f-41f9-82a1-0bddaaa6df8a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.322518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712da33-6c2f-41f9-82a1-0bddaaa6df8a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.322992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712da33-6c2f-41f9-82a1-0bddaaa6df8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.323981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712da33-6c2f-41f9-82a1-0bddaaa6df8a None admin] RBAC: Adding query filter params (interface=public, service_id=a2e176a30661489faf34cfb7c1dc6167, region_id=RegionOne) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:07.324524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712da33-6c2f-41f9-82a1-0bddaaa6df8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:07.326103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712da33-6c2f-41f9-82a1-0bddaaa6df8a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:07.331895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 465/924] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 25 05:42:07 2022] GET /identity/v3/endpoints?service_id=a2e176a30661489faf34cfb7c1dc6167&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:07.338397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:07.422878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:07.447455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e0f9d-aaa9-4613-bf56-ee623a96df81 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:07.486071 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748b3a49-b32d-4e3a-ac80-3764284503b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:07.487126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b3a49-b32d-4e3a-ac80-3764284503b3 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.487622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b3a49-b32d-4e3a-ac80-3764284503b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.488167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b3a49-b32d-4e3a-ac80-3764284503b3 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.489589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b3a49-b32d-4e3a-ac80-3764284503b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:07.491171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748b3a49-b32d-4e3a-ac80-3764284503b3 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:07.498145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 460/925] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:42:07 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:42:07.877258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607382c-8bb5-43cc-b646-f45449f1de4f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.877546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607382c-8bb5-43cc-b646-f45449f1de4f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.877828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607382c-8bb5-43cc-b646-f45449f1de4f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.880013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 466/926] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:07.885321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1dfd-498e-4e9d-9f80-f446231f8b59 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:07.885799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1dfd-498e-4e9d-9f80-f446231f8b59 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:07.885888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe1dfd-498e-4e9d-9f80-f446231f8b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:07.902942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-20fe1dfd-498e-4e9d-9f80-f446231f8b59 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:07.904138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-20fe1dfd-498e-4e9d-9f80-f446231f8b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:07.980864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 461/927] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:08.004142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607382c-8bb5-43cc-b646-f45449f1de4f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:08.108899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607382c-8bb5-43cc-b646-f45449f1de4f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:08.143921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:08.145637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.146555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.147048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.147969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:08.149720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:08.151328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING keystone.server.flask.common [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 25 05:42:08.167801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:08.207311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.sql.core [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:42:08.207311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 05:42:08.207311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [parameters: {'id': '3c43f193f2c04a528672db1a0d768ce9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 05:42:08.207311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86792) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-91a155a9-08c6-4fd6-8d00-e3d2a4fdd61d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3c43f193f2c04a528672db1a0d768ce9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3c43f193f2c04a528672db1a0d768ce9. Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:42:08.227743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result.read() Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 25 05:42:08.228660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application yield resource Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise self.value Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application session.commit() Jul 25 05:42:08.229628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:42:08.230529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:42:08.231406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result.read() Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:42:08.232221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [parameters: {'id': '3c43f193f2c04a528672db1a0d768ce9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:08.233094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:08.233887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3c43f193f2c04a528672db1a0d768ce9. Jul 25 05:42:08.237310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.240788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 467/928] 10.222.0.40 () {60 vars in 1140 bytes} [Mon Jul 25 05:42:08 2022] POST /identity/v3/projects => generated 262 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:08.246423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe1dfd-498e-4e9d-9f80-f446231f8b59 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:08.324738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:08.353252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe1dfd-498e-4e9d-9f80-f446231f8b59 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:08.383752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584b5e-6dcf-4362-9988-538642825030 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.384010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584b5e-6dcf-4362-9988-538642825030 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.384255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584b5e-6dcf-4362-9988-538642825030 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.387883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:08.388723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.389006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.389286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.392306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 468/929] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:08.393838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19267c79-ef47-4d1a-94a8-ff1939e7924f None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.394128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19267c79-ef47-4d1a-94a8-ff1939e7924f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.394409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19267c79-ef47-4d1a-94a8-ff1939e7924f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.397303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:08.399281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:42:08.416735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:08.417861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:08.418976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:42:08.418976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:42:08.418976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:08.418976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:08.418976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:08.418976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:08.418976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:08.418976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:08.419504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-19267c79-ef47-4d1a-94a8-ff1939e7924f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:08.419504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-19267c79-ef47-4d1a-94a8-ff1939e7924f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:08.421427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 462/930] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:08.427366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:08.506413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 469/931] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:08.525373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19267c79-ef47-4d1a-94a8-ff1939e7924f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:08.536766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c9592-b593-4c33-8600-065e33594aaa None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:08.567920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:08.573280 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:08.573954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.574241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.574520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.575280 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:08.575591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:08.577059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:08.604246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 463/932] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:42:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:08.642774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19267c79-ef47-4d1a-94a8-ff1939e7924f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:08.667705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:08.689630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1ec50-4d0c-48c7-8d40-a266ec3be72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:08.690725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1ec50-4d0c-48c7-8d40-a266ec3be72a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.691205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1ec50-4d0c-48c7-8d40-a266ec3be72a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.691635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1ec50-4d0c-48c7-8d40-a266ec3be72a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.693226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1ec50-4d0c-48c7-8d40-a266ec3be72a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:08.695262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1ec50-4d0c-48c7-8d40-a266ec3be72a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-27f1ec50-4d0c-48c7-8d40-a266ec3be72a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:08.709265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:08.710455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 25 05:42:08.711500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:08.718837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 470/933] 10.222.0.40 () {56 vars in 1132 bytes} [Mon Jul 25 05:42:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:08.721656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:08.763963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbb1aa-c9ca-4093-8bc7-cd26922956a9 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.764303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbb1aa-c9ca-4093-8bc7-cd26922956a9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.764600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbb1aa-c9ca-4093-8bc7-cd26922956a9 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.766148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 471/934] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:08.771946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7774cf-a4ce-4124-aafe-adeedca8868d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.772280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7774cf-a4ce-4124-aafe-adeedca8868d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.772564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7774cf-a4ce-4124-aafe-adeedca8868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.791934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9b7774cf-a4ce-4124-aafe-adeedca8868d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:08.795595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9b7774cf-a4ce-4124-aafe-adeedca8868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:08.865832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b94ded-ec0b-4399-bf7a-054f2f77027f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:08.881984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 472/935] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:08.907669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7774cf-a4ce-4124-aafe-adeedca8868d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:08.916191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:08.916191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:08.916191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:08.916191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:08.916949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:08.916949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:08.917331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:08.923935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 464/936] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:08.930124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.001666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:09.021869 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7774cf-a4ce-4124-aafe-adeedca8868d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.040566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:09.052514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3524b7-3ef2-41a7-810f-11ac626d8449 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.059577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351307b2-e850-49ad-b00d-e7a04e36ca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.060652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351307b2-e850-49ad-b00d-e7a04e36ca38 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.061043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351307b2-e850-49ad-b00d-e7a04e36ca38 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.061433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351307b2-e850-49ad-b00d-e7a04e36ca38 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.073726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351307b2-e850-49ad-b00d-e7a04e36ca38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-351307b2-e850-49ad-b00d-e7a04e36ca38 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:09.083686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:09.084928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:09.086081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:09.090526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 473/937] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 25 05:42:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:09.091774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351307b2-e850-49ad-b00d-e7a04e36ca38 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.094388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.095284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.095810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.096347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.097288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:09.097811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:09.099732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:09.109902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 465/938] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:09.111290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.202620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351307b2-e850-49ad-b00d-e7a04e36ca38 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.216152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd19eb-fe40-47ce-bcc5-bb58b5f869bf None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.236713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:09.241066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.241924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.242396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.242841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.243754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:09.244304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:09.245920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:09.252159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8abf6f-01b9-4533-b773-bc4c1cc57425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.252928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8abf6f-01b9-4533-b773-bc4c1cc57425 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.253284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8abf6f-01b9-4533-b773-bc4c1cc57425 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.253719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8abf6f-01b9-4533-b773-bc4c1cc57425 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.255155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8abf6f-01b9-4533-b773-bc4c1cc57425 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:09.256669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8abf6f-01b9-4533-b773-bc4c1cc57425 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:09.260377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 474/939] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 25 05:42:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:09.263384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.304452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:09.366456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 466/940] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:42:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:42:09.375550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2b020-2995-4918-a738-1951c561bad1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.424412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.425249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.425536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.425817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.438516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:09.440010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:09.461882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:09.462817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:09.463705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:09.463705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:09.463705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:09.463705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:09.463705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 475/941] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:09.469050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8abf6f-01b9-4533-b773-bc4c1cc57425 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.585264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8abf6f-01b9-4533-b773-bc4c1cc57425 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.620754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785d8f7-5493-49ab-aceb-d7781554137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.621432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785d8f7-5493-49ab-aceb-d7781554137d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.621723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785d8f7-5493-49ab-aceb-d7781554137d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.622001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785d8f7-5493-49ab-aceb-d7781554137d None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.622737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785d8f7-5493-49ab-aceb-d7781554137d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:09.623046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785d8f7-5493-49ab-aceb-d7781554137d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:09.624461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785d8f7-5493-49ab-aceb-d7781554137d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:09.638238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 467/942] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:09.644007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.726024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21651-0e41-4641-afeb-04b555e7ef94 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.726340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21651-0e41-4641-afeb-04b555e7ef94 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.726520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd21651-0e41-4641-afeb-04b555e7ef94 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.728041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 468/943] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:42:09.733512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9452a1f-2604-4a57-9252-2d69737798e8 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.733807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9452a1f-2604-4a57-9252-2d69737798e8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.734099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9452a1f-2604-4a57-9252-2d69737798e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.734304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:09.751276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ad2e0-d978-4a46-b103-bc723dad6479 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.751608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d9452a1f-2604-4a57-9252-2d69737798e8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:09.752839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d9452a1f-2604-4a57-9252-2d69737798e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:09.784801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:09.792330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.793151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.793436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.793770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.802445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:09.803795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:42:09.820850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:09.822043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:09.823109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 476/944] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:42:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:09.830064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.836240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 469/945] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:09.859035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9452a1f-2604-4a57-9252-2d69737798e8 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:09.951700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab02d2-361a-45f3-a6be-fdc843c382ca None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.990755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9452a1f-2604-4a57-9252-2d69737798e8 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:09.995038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:09.995729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:09.996006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:09.996307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:09.997156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:09.998035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:09.999493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.024556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 477/946] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 25 05:42:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:10.030204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.033234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:10.036580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.037661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.038199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.038688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.039600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.041595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:10.057132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:10.058220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:10.059074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:10.063407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 470/947] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:10.064751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.101120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:10.171264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc2cfd-523d-4d53-85a7-2f1c3742795d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.186122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda28408-1cc8-4a8a-b3c7-93fd3a569a4f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.210944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.214194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.214194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.214512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.230261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.231475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.231980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.232574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.233486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:10.233987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.235528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.237066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.239357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.242644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 471/948] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:10.248620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:10.278155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:10.279107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:10.280019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:10.280019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:10.280019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:10.280019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:10.284838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 478/949] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:10.287206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.365885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:10.379866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:10.390675 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e81617-a806-43a4-8889-7b1cff84a2a0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.431609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fdd40-e560-4ff3-82ba-c900121c5d05 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.436507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.437188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.437472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.437751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.438470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:10.438778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.440173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.453586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 472/950] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:10.455238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.485629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.486979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.487473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.488244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.489700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:10.490521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.492993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.515310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 479/951] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:10.517492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.603484 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ade48-3cc3-4705-b040-06c35aa4ec0b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.650649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0feb2-96d8-40e0-8917-592ee9e7fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.651328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0feb2-96d8-40e0-8917-592ee9e7fe35 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.653037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0feb2-96d8-40e0-8917-592ee9e7fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.653316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0feb2-96d8-40e0-8917-592ee9e7fe35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.654067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0feb2-96d8-40e0-8917-592ee9e7fe35 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.655451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0feb2-96d8-40e0-8917-592ee9e7fe35 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.662753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e017bb-eeab-4d71-b9b0-af0c961bd6e4 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.684493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:10.706209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3083013-7b49-49af-8976-53fc68877019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.707139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3083013-7b49-49af-8976-53fc68877019 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.708128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3083013-7b49-49af-8976-53fc68877019 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.708589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3083013-7b49-49af-8976-53fc68877019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.715752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3083013-7b49-49af-8976-53fc68877019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.717242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3083013-7b49-49af-8976-53fc68877019 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.733483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 473/952] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:42:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-f3083013-7b49-49af-8976-53fc68877019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:42:10.740055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:10.741383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:10.742609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:42:10.742609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:42:10.742609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:10.742609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:10.742609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:10.742609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:10.742609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:10.742609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:10.744687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 480/953] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:10.755393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0feb2-96d8-40e0-8917-592ee9e7fe35 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.782268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:10.898896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0feb2-96d8-40e0-8917-592ee9e7fe35 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:10.939504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:10.940470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.941026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.941305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.942965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:10.943254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:10.945074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:10.960434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 474/954] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:10.964781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:10.967558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8a92-6ecc-4792-89e5-ff7a819d1e0b None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.968236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8a92-6ecc-4792-89e5-ff7a819d1e0b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.968799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e8a92-6ecc-4792-89e5-ff7a819d1e0b None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:10.970561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 481/955] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:10.984096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae397d1-2386-4496-b155-feea7b9155cc None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:10.984698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae397d1-2386-4496-b155-feea7b9155cc None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:10.985039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae397d1-2386-4496-b155-feea7b9155cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.006702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4ae397d1-2386-4496-b155-feea7b9155cc None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:11.009093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4ae397d1-2386-4496-b155-feea7b9155cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:11.059369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:11.088267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 482/956] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:11.103914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d047-69d1-43dd-9432-f84a3d776c3d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:11.115792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae397d1-2386-4496-b155-feea7b9155cc None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:11.140327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8258196-8f16-4ebd-8286-70116c42d72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:11.141635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8258196-8f16-4ebd-8286-70116c42d72c None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:11.142275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8258196-8f16-4ebd-8286-70116c42d72c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:11.142749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8258196-8f16-4ebd-8286-70116c42d72c None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/2cdbdedd9ba34e81916d5104157dfd10/roles/1c5a669f88fd44399a230e4f91a6c965` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.145791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8258196-8f16-4ebd-8286-70116c42d72c None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=2cdbdedd9ba34e81916d5104157dfd10, role_id=1c5a669f88fd44399a230e4f91a6c965)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:11.147465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8258196-8f16-4ebd-8286-70116c42d72c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:11.220346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 475/957] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:42:10 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/2cdbdedd9ba34e81916d5104157dfd10/roles/1c5a669f88fd44399a230e4f91a6c965 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:11.221110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae397d1-2386-4496-b155-feea7b9155cc None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:11.225051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:11.262132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:11.262748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:11.263035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:11.263316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.264050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:11.270792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:11.338685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.sql.core [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:42:11.338685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 05:42:11.338685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [parameters: {'user_id': '2ac069e7daeb4a46bad609759a96ed05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 05:42:11.338685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86792) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result.read() Jul 25 05:42:11.346927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:42:11.348541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application yield resource Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application next(self.gen) Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.session.rollback() Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.force_reraise() Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise self.value Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application session.commit() Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 25 05:42:11.349626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._prepare_impl() Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.session.flush() Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._flush(objects) Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application compat.raise_( Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application flush_context.execute() Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rec.execute(self) Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 25 05:42:11.350533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 25 05:42:11.351443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result = self._query(query) Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application conn.query(q) Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application result.read() Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 25 05:42:11.353372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [parameters: {'user_id': '2ac069e7daeb4a46bad609759a96ed05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:11.355466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 25 05:42:11.357468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 25 05:42:11.359739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 25 05:42:11.359739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 25 05:42:11.359739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 25 05:42:11.359739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 25 05:42:11.359739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 25 05:42:11.359739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:11.359739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 483/958] 10.222.0.40 () {60 vars in 1131 bytes} [Mon Jul 25 05:42:11 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:11.368131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8258196-8f16-4ebd-8286-70116c42d72c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:11.475897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8258196-8f16-4ebd-8286-70116c42d72c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:11.505659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:11.511840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86b3d6-804d-4d02-b02e-02181625e0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:11.512745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86b3d6-804d-4d02-b02e-02181625e0f1 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:11.513111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86b3d6-804d-4d02-b02e-02181625e0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:11.513393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86b3d6-804d-4d02-b02e-02181625e0f1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.522077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86b3d6-804d-4d02-b02e-02181625e0f1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:11.526703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86b3d6-804d-4d02-b02e-02181625e0f1 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-7d86b3d6-804d-4d02-b02e-02181625e0f1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:42:11.537986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:11.538916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:11.539921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 476/959] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:11.545816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:11.658844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e71ee-abbe-4c82-9f77-ee75335cc429 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:11.672153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec54750-4ffb-4f72-b120-8b7c7325325c None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:11.672733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec54750-4ffb-4f72-b120-8b7c7325325c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:11.673228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec54750-4ffb-4f72-b120-8b7c7325325c None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.675137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 477/960] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:42:11.681314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f167f-a9e6-43e6-a4b2-33669bb4f10f None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:11.681982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f167f-a9e6-43e6-a4b2-33669bb4f10f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:11.682466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f167f-a9e6-43e6-a4b2-33669bb4f10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.697075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:11.697757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:11.698042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:11.698322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.699137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:11.699450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:11.701141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:11.703542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-659f167f-a9e6-43e6-a4b2-33669bb4f10f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:11.705044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-659f167f-a9e6-43e6-a4b2-33669bb4f10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:11.740192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 484/961] 10.222.0.40 () {56 vars in 1101 bytes} [Mon Jul 25 05:42:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:11.795589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 478/962] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:11.819306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:11.848519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:11.941216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15cd3f-df60-4b72-a55a-825c1f5209fc None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:11.981348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:11.982166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:11.982450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:11.982726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] PATH_INFO: `/v3/services/compute` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:11.983401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:11.984874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:11.998858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:11.999961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:12.000847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 485/963] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:12.007723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f167f-a9e6-43e6-a4b2-33669bb4f10f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.116677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f167f-a9e6-43e6-a4b2-33669bb4f10f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.137185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:12.151008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39abc064-3311-4855-bd6a-adc207fd2852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.151678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abc064-3311-4855-bd6a-adc207fd2852 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.152140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abc064-3311-4855-bd6a-adc207fd2852 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.152365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39abc064-3311-4855-bd6a-adc207fd2852 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.153123 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abc064-3311-4855-bd6a-adc207fd2852 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:12.153438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abc064-3311-4855-bd6a-adc207fd2852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.154808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39abc064-3311-4855-bd6a-adc207fd2852 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.159801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 479/964] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:42:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:12.166445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:12.258784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:12.271526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75792f-73b5-46b9-b024-06524ab72b3f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:12.308064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b1fc6-05e0-4385-8621-a94de3527b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:12.308746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b1fc6-05e0-4385-8621-a94de3527b27 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.309026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b1fc6-05e0-4385-8621-a94de3527b27 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.309294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b1fc6-05e0-4385-8621-a94de3527b27 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.309993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b1fc6-05e0-4385-8621-a94de3527b27 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:12.310295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b1fc6-05e0-4385-8621-a94de3527b27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:12.311644 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b1fc6-05e0-4385-8621-a94de3527b27 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:12.316499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 486/965] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:42:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:12.949547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dbeaa-86c2-4b70-9bb6-aa174da1b4d1 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.949837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dbeaa-86c2-4b70-9bb6-aa174da1b4d1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.950117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dbeaa-86c2-4b70-9bb6-aa174da1b4d1 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.951527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 480/966] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:12.956947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be11005-0ab3-4f90-bb4b-ccc0ffe735b0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:12.957872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be11005-0ab3-4f90-bb4b-ccc0ffe735b0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:12.958171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be11005-0ab3-4f90-bb4b-ccc0ffe735b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:12.975454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2be11005-0ab3-4f90-bb4b-ccc0ffe735b0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:12.977008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2be11005-0ab3-4f90-bb4b-ccc0ffe735b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:13.048338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 487/967] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:13.068734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dbeaa-86c2-4b70-9bb6-aa174da1b4d1 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.175324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dbeaa-86c2-4b70-9bb6-aa174da1b4d1 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.210291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43f3a3-df6e-4d1f-aea7-81653a90a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.211096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f3a3-df6e-4d1f-aea7-81653a90a947 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.211376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f3a3-df6e-4d1f-aea7-81653a90a947 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.211646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43f3a3-df6e-4d1f-aea7-81653a90a947 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.212324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43f3a3-df6e-4d1f-aea7-81653a90a947 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:13.213695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43f3a3-df6e-4d1f-aea7-81653a90a947 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-7a43f3a3-df6e-4d1f-aea7-81653a90a947 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:13.226625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:13.227929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:13.229911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:13.232578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 481/968] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:13.233798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be11005-0ab3-4f90-bb4b-ccc0ffe735b0 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.314633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:13.333137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be11005-0ab3-4f90-bb4b-ccc0ffe735b0 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.366809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.367551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.367832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.368104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.368852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:13.369159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:13.369282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520fff1-5fdd-4ced-939d-4e82393ce7af None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.369529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520fff1-5fdd-4ced-939d-4e82393ce7af None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.369814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520fff1-5fdd-4ced-939d-4e82393ce7af None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.370536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:13.371310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 482/969] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:13.376219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03821d8c-7c0a-4101-bf40-698c467dd74b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.377347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03821d8c-7c0a-4101-bf40-698c467dd74b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.377482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 488/970] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:13.377576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03821d8c-7c0a-4101-bf40-698c467dd74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.382490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.394296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-03821d8c-7c0a-4101-bf40-698c467dd74b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:13.396307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-03821d8c-7c0a-4101-bf40-698c467dd74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:13.442645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:13.466847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 483/971] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:13.491572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03821d8c-7c0a-4101-bf40-698c467dd74b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.494065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212d391-08eb-496e-99d4-c48dd1beef5f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.532274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.532948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.533230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.533506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.534219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:13.534527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:13.536643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:13.541877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 489/972] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:13.547285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.560810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:13.605059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03821d8c-7c0a-4101-bf40-698c467dd74b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.640395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9c3f7-955e-46a9-81a3-1073468b0ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.641202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9c3f7-955e-46a9-81a3-1073468b0ca1 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.641486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9c3f7-955e-46a9-81a3-1073468b0ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.641765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb9c3f7-955e-46a9-81a3-1073468b0ca1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.652196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb9c3f7-955e-46a9-81a3-1073468b0ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:13.655586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c31d562-7174-4611-b8f2-4a6462fe7a3e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-2cb9c3f7-955e-46a9-81a3-1073468b0ca1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:13.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:13.662179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:13.663381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 484/973] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:13.668025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9c3f7-955e-46a9-81a3-1073468b0ca1 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.699541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.700233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.700542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.700808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.701607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] RBAC: Adding query filter params (interface=admin, service_id=a2e176a30661489faf34cfb7c1dc6167, region_id=RegionOne) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:13.701922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:13.703286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:13.713952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 490/974] 10.222.0.40 () {56 vars in 1249 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/endpoints?service_id=a2e176a30661489faf34cfb7c1dc6167&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:13.714101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:13.715280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.781190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb9c3f7-955e-46a9-81a3-1073468b0ca1 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.800641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:13.822370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.823180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.823547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.823898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.824763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:13.825223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:13.826649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:13.827020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea965c7-ddb6-42e9-8ac4-7a36b2c96655 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:13.836474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 485/975] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:13.841088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:13.865950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd26fb7-74f7-4ca4-be4c-6e18338783cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:13.866708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd26fb7-74f7-4ca4-be4c-6e18338783cb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:13.866993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd26fb7-74f7-4ca4-be4c-6e18338783cb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:13.867268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd26fb7-74f7-4ca4-be4c-6e18338783cb None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:13.867958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd26fb7-74f7-4ca4-be4c-6e18338783cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:13.869498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd26fb7-74f7-4ca4-be4c-6e18338783cb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:13.884149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 491/976] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:42:13.958295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:13.962161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57128d-0ef6-4567-8786-0280c67e3e02 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:14.001564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:14.002815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.003200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.003604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.015750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:14.017418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:14.036376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:14.037514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:14.038656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:14.038656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:14.038656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:14.038656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:14.040070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 486/977] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:14.045974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd26fb7-74f7-4ca4-be4c-6e18338783cb None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:14.152790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd26fb7-74f7-4ca4-be4c-6e18338783cb None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:14.188003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581116d-657b-4fb1-9dc4-9fae6f6f9135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:14.188699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581116d-657b-4fb1-9dc4-9fae6f6f9135 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.188984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581116d-657b-4fb1-9dc4-9fae6f6f9135 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.189263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581116d-657b-4fb1-9dc4-9fae6f6f9135 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.189995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581116d-657b-4fb1-9dc4-9fae6f6f9135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:14.190302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581116d-657b-4fb1-9dc4-9fae6f6f9135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:14.191708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581116d-657b-4fb1-9dc4-9fae6f6f9135 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:14.198913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:14.204845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 492/978] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:14.210701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:14.312231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14e695-dc5a-4f1d-9780-1862f89c15b6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:14.347382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21888f6-caf1-4e5d-958d-65199f0c5903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:14.348956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21888f6-caf1-4e5d-958d-65199f0c5903 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.349185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21888f6-caf1-4e5d-958d-65199f0c5903 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.349465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21888f6-caf1-4e5d-958d-65199f0c5903 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.356717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21888f6-caf1-4e5d-958d-65199f0c5903 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:14.358236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21888f6-caf1-4e5d-958d-65199f0c5903 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b21888f6-caf1-4e5d-958d-65199f0c5903 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:42:14.374241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:14.375419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 25 05:42:14.376637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:14.377316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccbda2-5c6d-41b1-9a15-576ad8885e76 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.377316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccbda2-5c6d-41b1-9a15-576ad8885e76 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.377316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ccbda2-5c6d-41b1-9a15-576ad8885e76 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.380146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 493/979] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:14.380294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 487/980] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:42:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:14.381010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccbda2-5c6d-41b1-9a15-576ad8885e76 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:14.383373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce218ce4-23cd-4332-8190-a643b59365f3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.383664 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce218ce4-23cd-4332-8190-a643b59365f3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.383945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce218ce4-23cd-4332-8190-a643b59365f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.400917 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ce218ce4-23cd-4332-8190-a643b59365f3 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:14.402110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ce218ce4-23cd-4332-8190-a643b59365f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:14.410776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:14.471906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 488/981] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:14.495942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce218ce4-23cd-4332-8190-a643b59365f3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:14.501186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ccbda2-5c6d-41b1-9a15-576ad8885e76 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:14.536990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:14.537763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.538057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.538338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.539182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:14.539502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:14.540975 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:14.559277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 494/982] 10.222.0.40 () {56 vars in 1149 bytes} [Mon Jul 25 05:42:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:14.564801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:14.604086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce218ce4-23cd-4332-8190-a643b59365f3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:14.622715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:14.641406 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a81223-c941-4595-901a-d0a79eed81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:14.642167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a81223-c941-4595-901a-d0a79eed81de None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.642518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a81223-c941-4595-901a-d0a79eed81de None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.642861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a81223-c941-4595-901a-d0a79eed81de None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.643654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a81223-c941-4595-901a-d0a79eed81de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:14.645783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a81223-c941-4595-901a-d0a79eed81de None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:14.668348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a74be-4ae3-4c33-9f5c-2158f6bbfe41 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:14.704168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 489/983] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:42:14 2022] POST /identity/v3/services => generated 242 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:42:14.718073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c4672-69ae-460b-b045-e30e38d9aeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:14.718921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c4672-69ae-460b-b045-e30e38d9aeac None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:14.719306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c4672-69ae-460b-b045-e30e38d9aeac None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:14.719685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c4672-69ae-460b-b045-e30e38d9aeac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:14.800448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c4672-69ae-460b-b045-e30e38d9aeac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:14.802152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c4672-69ae-460b-b045-e30e38d9aeac None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:14.859880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:14.909511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 495/984] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:14 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:42:14.915998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a81223-c941-4595-901a-d0a79eed81de None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:14.918987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:15.022501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a81223-c941-4595-901a-d0a79eed81de None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:15.058518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:15.059325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:15.059610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:15.059888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:15.071232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:15.072724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:15.090075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:15.091054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:15.091957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:15.091957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:15.091957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:15.091957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:15.091957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 490/985] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:15.097074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c4672-69ae-460b-b045-e30e38d9aeac None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:15.199464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c4672-69ae-460b-b045-e30e38d9aeac None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:15.234445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:15.235114 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:15.235400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:15.235677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:15.236434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:15.236745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:15.238115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:15.254913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 496/986] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:15.256311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:15.312152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:15.357071 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274d9cb-7fce-450e-bd24-a8d2e277252b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:15.390979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:15.391775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:15.392058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:15.392359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:15.399148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:15.400619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:42:15.414875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:15.415798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:15.421687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 491/987] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:15.423696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:15.440091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:15.529065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27822100-4b82-4354-95db-c6b950a9948e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:15.563766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884670f-bdca-4137-bbdc-c547301b750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:15.564473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884670f-bdca-4137-bbdc-c547301b750e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:15.564760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884670f-bdca-4137-bbdc-c547301b750e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:15.565044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884670f-bdca-4137-bbdc-c547301b750e None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:15.565839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884670f-bdca-4137-bbdc-c547301b750e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:15.566212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884670f-bdca-4137-bbdc-c547301b750e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:15.568393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884670f-bdca-4137-bbdc-c547301b750e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:15.580748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 497/988] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:15.586889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:15.692249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875534e9-0525-4849-83c7-e8a2556d38ad None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:15.708062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:15.729308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be5d9a-fbc3-44e2-946a-1c73b804100b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:15.729973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be5d9a-fbc3-44e2-946a-1c73b804100b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:15.730306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be5d9a-fbc3-44e2-946a-1c73b804100b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:15.730593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be5d9a-fbc3-44e2-946a-1c73b804100b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:15.731426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be5d9a-fbc3-44e2-946a-1c73b804100b None admin] RBAC: Adding query filter params (role.id=1c5a669f88fd44399a230e4f91a6c965, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=2cdbdedd9ba34e81916d5104157dfd10) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:15.731742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be5d9a-fbc3-44e2-946a-1c73b804100b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:15.733269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be5d9a-fbc3-44e2-946a-1c73b804100b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:15.740263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 492/989] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:15 2022] GET /identity/v3/role_assignments?user.id=2cdbdedd9ba34e81916d5104157dfd10&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=1c5a669f88fd44399a230e4f91a6c965 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:15.966452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d203d-c658-4241-9e52-d4f0abc4e71a None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:15.966851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d203d-c658-4241-9e52-d4f0abc4e71a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:15.967048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d203d-c658-4241-9e52-d4f0abc4e71a None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:15.968562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 498/990] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:15.973738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dedf3e-7fdb-48bc-958c-c5cafe08b862 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:15.974029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dedf3e-7fdb-48bc-958c-c5cafe08b862 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:15.974313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dedf3e-7fdb-48bc-958c-c5cafe08b862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:15.990837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-66dedf3e-7fdb-48bc-958c-c5cafe08b862 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:15.992023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-66dedf3e-7fdb-48bc-958c-c5cafe08b862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:16.059957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 493/991] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:16.083633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d203d-c658-4241-9e52-d4f0abc4e71a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:16.156349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:16.186558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d203d-c658-4241-9e52-d4f0abc4e71a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:16.221245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:16.221967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:16.222253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:16.222526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:16.223276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:16.224684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:16.236758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:16.237768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:16.239295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 499/992] 10.222.0.40 () {56 vars in 1126 bytes} [Mon Jul 25 05:42:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:16.246219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dedf3e-7fdb-48bc-958c-c5cafe08b862 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:16.346516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dedf3e-7fdb-48bc-958c-c5cafe08b862 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:16.356772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:16.381730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:16.382400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:16.382686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:16.382961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:16.383665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:16.383971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:16.385389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:16.390556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 494/993] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:16.396787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:16.501032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208492fd-615f-4a6a-95c6-74bd4fae0386 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:16.535841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397eb19f-606e-4421-ba1e-7964214815b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:16.536769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb19f-606e-4421-ba1e-7964214815b6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:16.537052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb19f-606e-4421-ba1e-7964214815b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:16.537328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397eb19f-606e-4421-ba1e-7964214815b6 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:16.538037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb19f-606e-4421-ba1e-7964214815b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:16.538344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb19f-606e-4421-ba1e-7964214815b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:16.539710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397eb19f-606e-4421-ba1e-7964214815b6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:16.541871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:16.550223 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 500/994] 10.222.0.40 () {56 vars in 1122 bytes} [Mon Jul 25 05:42:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:16.552252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:16.653445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74809b-9cda-4a77-a2f5-7fa38b15589f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:16.687770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960c3f4-01ac-41bf-86f4-f32e80c07280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:16.688464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960c3f4-01ac-41bf-86f4-f32e80c07280 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:16.688753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960c3f4-01ac-41bf-86f4-f32e80c07280 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:16.689033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960c3f4-01ac-41bf-86f4-f32e80c07280 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:16.689786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960c3f4-01ac-41bf-86f4-f32e80c07280 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:16.691146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960c3f4-01ac-41bf-86f4-f32e80c07280 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:16.749928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 495/995] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:42:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:16.774857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c84c6c-6f1a-454e-bb5e-9170e0c5d5eb None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:16.775326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c84c6c-6f1a-454e-bb5e-9170e0c5d5eb None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:16.775707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c84c6c-6f1a-454e-bb5e-9170e0c5d5eb None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:16.777320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 501/996] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:16.783371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ceeeb-f07a-4a5c-ac7f-9fa7a1051012 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:16.783663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ceeeb-f07a-4a5c-ac7f-9fa7a1051012 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:16.783941 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ceeeb-f07a-4a5c-ac7f-9fa7a1051012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:16.801410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d63ceeeb-f07a-4a5c-ac7f-9fa7a1051012 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:16.802740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d63ceeeb-f07a-4a5c-ac7f-9fa7a1051012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:16.832858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:16.920008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 496/997] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:16.941948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c84c6c-6f1a-454e-bb5e-9170e0c5d5eb None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:17.045461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c84c6c-6f1a-454e-bb5e-9170e0c5d5eb None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:17.079242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:17.080035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.080348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.080624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.081276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:17.083302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:17.095055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:17.096310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:17.097382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 502/998] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:17.103482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ceeeb-f07a-4a5c-ac7f-9fa7a1051012 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:17.202347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ceeeb-f07a-4a5c-ac7f-9fa7a1051012 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:17.238745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:17.239490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.239777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.240054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.240797 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:17.241142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:17.242525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:17.247672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 497/999] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:42:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:17.253310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:17.318378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:17.358530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399665ce-8252-4c7b-b192-6a879cfaa172 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:17.392312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:17.392979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.393263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.393542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.394247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] RBAC: Adding query filter params (type=compute) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:17.394682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:17.396056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:17.405848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 503/1000] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:42:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:17.407131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:17.432511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:17.509880 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51329a5d-31b6-43a1-a53a-882800fc43f8 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:17.544314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff0afb-2b5b-49da-a092-2220990c698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:17.544980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff0afb-2b5b-49da-a092-2220990c698b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.545265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff0afb-2b5b-49da-a092-2220990c698b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.545543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff0afb-2b5b-49da-a092-2220990c698b None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.546327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff0afb-2b5b-49da-a092-2220990c698b None admin] RBAC: Adding query filter params (interface=public, service_id=397038fffd904f0e8dbc47717366e826, region_id=RegionOne) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:17.546642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff0afb-2b5b-49da-a092-2220990c698b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:17.548015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff0afb-2b5b-49da-a092-2220990c698b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:17.558290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 498/1001] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 25 05:42:17 2022] GET /identity/v3/endpoints?service_id=397038fffd904f0e8dbc47717366e826&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:17.559207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:17.660487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8eaf54-125d-47ce-aa93-52581ea10c2c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:17.691378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:17.702011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38b7c7-488d-4e17-94f0-52a47f948802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:17.703626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38b7c7-488d-4e17-94f0-52a47f948802 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.703626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38b7c7-488d-4e17-94f0-52a47f948802 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.703626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38b7c7-488d-4e17-94f0-52a47f948802 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.703995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38b7c7-488d-4e17-94f0-52a47f948802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:17.705397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38b7c7-488d-4e17-94f0-52a47f948802 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:17.720215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 504/1002] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:42:17 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:42:17.831956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9a1d7-c275-4b73-97fb-db735d191505 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.832742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9a1d7-c275-4b73-97fb-db735d191505 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.833091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9a1d7-c275-4b73-97fb-db735d191505 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.834708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 499/1003] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:17.840578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191057b-915d-4d54-bae6-935c9ea07c89 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:17.840863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191057b-915d-4d54-bae6-935c9ea07c89 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:17.841143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191057b-915d-4d54-bae6-935c9ea07c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:17.859946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f191057b-915d-4d54-bae6-935c9ea07c89 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:17.861166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f191057b-915d-4d54-bae6-935c9ea07c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:17.930641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 505/1004] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:17 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:17.953171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9a1d7-c275-4b73-97fb-db735d191505 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.039129 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:18.054865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9a1d7-c275-4b73-97fb-db735d191505 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.090559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.091452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.091802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.092165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] PATH_INFO: `/v3/services/image` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.092886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.094306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:18.105778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:18.107006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:18.108088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:18.113979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 500/1005] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Jul 25 05:42:17 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:18.115257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191057b-915d-4d54-bae6-935c9ea07c89 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.216632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191057b-915d-4d54-bae6-935c9ea07c89 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.238702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:18.250228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaf310-d9bf-4fb8-85a4-cfad721d5262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.250904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf310-d9bf-4fb8-85a4-cfad721d5262 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.258490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf310-d9bf-4fb8-85a4-cfad721d5262 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.258490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf310-d9bf-4fb8-85a4-cfad721d5262 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.258490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaf310-d9bf-4fb8-85a4-cfad721d5262 None admin] RBAC: Adding query filter params (name=image) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:18.258490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaf310-d9bf-4fb8-85a4-cfad721d5262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.258490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaf310-d9bf-4fb8-85a4-cfad721d5262 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.259448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 506/1006] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:18.265012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:18.366304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f970f3f-9c6a-4b59-86fc-847ba33e8238 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:18.400495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3ad63b-38f3-4f40-82d5-7dd9909ca0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:18.401171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad63b-38f3-4f40-82d5-7dd9909ca0ef None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.401462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad63b-38f3-4f40-82d5-7dd9909ca0ef None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.401742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3ad63b-38f3-4f40-82d5-7dd9909ca0ef None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.402451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ad63b-38f3-4f40-82d5-7dd9909ca0ef None admin] RBAC: Adding query filter params (type=image) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:18.402759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ad63b-38f3-4f40-82d5-7dd9909ca0ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:18.404164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3ad63b-38f3-4f40-82d5-7dd9909ca0ef None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:18.408892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 501/1007] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:18.877948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e942e7-13f8-4190-813c-f00a3917b0fa None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.878459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e942e7-13f8-4190-813c-f00a3917b0fa None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.878537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e942e7-13f8-4190-813c-f00a3917b0fa None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.880702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 507/1008] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:18.885787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3207-b4be-42c0-a3dd-753e4a119edd None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:18.886082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3207-b4be-42c0-a3dd-753e4a119edd None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:18.886365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b3207-b4be-42c0-a3dd-753e4a119edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:18.903710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-125b3207-b4be-42c0-a3dd-753e4a119edd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:18.905016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-125b3207-b4be-42c0-a3dd-753e4a119edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:18.914698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:18.974017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 502/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:18 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:18.996944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e942e7-13f8-4190-813c-f00a3917b0fa None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:19.099057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e942e7-13f8-4190-813c-f00a3917b0fa None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:19.133778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7b287-3d21-4f83-aa14-c314d7ec9bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:19.134572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7b287-3d21-4f83-aa14-c314d7ec9bfb None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:19.134859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7b287-3d21-4f83-aa14-c314d7ec9bfb None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:19.135137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7b287-3d21-4f83-aa14-c314d7ec9bfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:19.136327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7b287-3d21-4f83-aa14-c314d7ec9bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:19.137761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7b287-3d21-4f83-aa14-c314d7ec9bfb None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:19.139533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 508/1010] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:19.145665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b3207-b4be-42c0-a3dd-753e4a119edd None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:19.244151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b3207-b4be-42c0-a3dd-753e4a119edd None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:19.277347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580200-edd9-4d58-afba-c796750706f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:19.278013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580200-edd9-4d58-afba-c796750706f6 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:19.278297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580200-edd9-4d58-afba-c796750706f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:19.278574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580200-edd9-4d58-afba-c796750706f6 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:19.279292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31580200-edd9-4d58-afba-c796750706f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:19.280757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31580200-edd9-4d58-afba-c796750706f6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:19.295098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:19.379781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 503/1011] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:42:19 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:19.792636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd238455-88fb-409a-925a-ffd119a9f512 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:19.792928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd238455-88fb-409a-925a-ffd119a9f512 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:19.793212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd238455-88fb-409a-925a-ffd119a9f512 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:19.794737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 509/1012] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:19.799802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ec4b3-62d4-40d1-bca4-029a80e36096 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:19.800341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ec4b3-62d4-40d1-bca4-029a80e36096 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:19.800420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ec4b3-62d4-40d1-bca4-029a80e36096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:19.817706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-333ec4b3-62d4-40d1-bca4-029a80e36096 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:19.818895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-333ec4b3-62d4-40d1-bca4-029a80e36096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:19.889491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 504/1013] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:19.912525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd238455-88fb-409a-925a-ffd119a9f512 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:19.969157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:20.014432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd238455-88fb-409a-925a-ffd119a9f512 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.048751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.049467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.049748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.050017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.050698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.052061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:20.063744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:20.065009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 25 05:42:20.065878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:20.067561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 510/1014] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:20.071329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ec4b3-62d4-40d1-bca4-029a80e36096 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:20.174236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ec4b3-62d4-40d1-bca4-029a80e36096 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.200063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:20.209848 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.210530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.210814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.220066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.220066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:20.220066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.220066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.220066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 505/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:42:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:20.224776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:20.329121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca61f27a-e25e-4daf-ad87-f66074a6e981 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.347064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:20.363750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09112975-30a8-4950-bf39-572d1ddb9df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.364488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112975-30a8-4950-bf39-572d1ddb9df8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.364774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112975-30a8-4950-bf39-572d1ddb9df8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.370439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112975-30a8-4950-bf39-572d1ddb9df8 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.370439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09112975-30a8-4950-bf39-572d1ddb9df8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:20.370439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09112975-30a8-4950-bf39-572d1ddb9df8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.370439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09112975-30a8-4950-bf39-572d1ddb9df8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.373080 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 511/1016] 10.222.0.40 () {56 vars in 1112 bytes} [Mon Jul 25 05:42:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:20.379082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:20.462763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93233730-84f8-4586-ac5a-ab233133422f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.463207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93233730-84f8-4586-ac5a-ab233133422f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.463317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93233730-84f8-4586-ac5a-ab233133422f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.464854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 512/1017] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:42:20.469473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad97e8-4059-49e6-b4c9-a3abc23aa2ae None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.469765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad97e8-4059-49e6-b4c9-a3abc23aa2ae None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.470051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad97e8-4059-49e6-b4c9-a3abc23aa2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.481084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e112715-a0c9-4c23-a35b-2144bab359b1 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.487044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4aad97e8-4059-49e6-b4c9-a3abc23aa2ae None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:20.488566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4aad97e8-4059-49e6-b4c9-a3abc23aa2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:20.518336 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ba0cd-5d95-420d-8891-7166a96dd087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.519006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ba0cd-5d95-420d-8891-7166a96dd087 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.519926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ba0cd-5d95-420d-8891-7166a96dd087 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.520243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ba0cd-5d95-420d-8891-7166a96dd087 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.521008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ba0cd-5d95-420d-8891-7166a96dd087 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.522377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ba0cd-5d95-420d-8891-7166a96dd087 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.560469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 513/1018] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:20.584504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad97e8-4059-49e6-b4c9-a3abc23aa2ae None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:20.613525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 506/1019] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:42:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:20.700874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad97e8-4059-49e6-b4c9-a3abc23aa2ae None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:20.715454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:20.744281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23ee5-38e4-4a63-b34d-0fb3f55a6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:20.744955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23ee5-38e4-4a63-b34d-0fb3f55a6912 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:20.745234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23ee5-38e4-4a63-b34d-0fb3f55a6912 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:20.745506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23ee5-38e4-4a63-b34d-0fb3f55a6912 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:20.746225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc23ee5-38e4-4a63-b34d-0fb3f55a6912 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:20.747629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc23ee5-38e4-4a63-b34d-0fb3f55a6912 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:20.825603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 514/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:42:20 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:42:21.478839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983c855-8ec0-4143-9177-675f26029369 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:21.479402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983c855-8ec0-4143-9177-675f26029369 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:21.479402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0983c855-8ec0-4143-9177-675f26029369 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:21.480811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 507/1021] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:21.486364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8fea-883f-40de-86d4-94792625288a None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:21.486755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8fea-883f-40de-86d4-94792625288a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:21.486950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bf8fea-883f-40de-86d4-94792625288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:21.503416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-01bf8fea-883f-40de-86d4-94792625288a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:21.504769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-01bf8fea-883f-40de-86d4-94792625288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:21.577017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 515/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:21.602250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983c855-8ec0-4143-9177-675f26029369 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:21.651251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:21.705590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0983c855-8ec0-4143-9177-675f26029369 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:21.740884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9479eb-3c15-4af9-a7c6-6262620abc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:21.741594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9479eb-3c15-4af9-a7c6-6262620abc72 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:21.741871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9479eb-3c15-4af9-a7c6-6262620abc72 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:21.742131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9479eb-3c15-4af9-a7c6-6262620abc72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:21.743287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9479eb-3c15-4af9-a7c6-6262620abc72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:21.744742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9479eb-3c15-4af9-a7c6-6262620abc72 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:21.750900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 508/1023] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:21.752530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf8fea-883f-40de-86d4-94792625288a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:21.849976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:21.856572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bf8fea-883f-40de-86d4-94792625288a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:21.892576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34147a88-b446-4ac5-9908-da79f4a141bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:21.893267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147a88-b446-4ac5-9908-da79f4a141bf None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:21.893554 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147a88-b446-4ac5-9908-da79f4a141bf None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:21.893835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147a88-b446-4ac5-9908-da79f4a141bf None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:21.895281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34147a88-b446-4ac5-9908-da79f4a141bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:21.896945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34147a88-b446-4ac5-9908-da79f4a141bf None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:21.978936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 516/1024] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:42:21 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:42:22.837781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f66e1-963d-40d3-a98e-aab01d60f4db None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:22.838293 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f66e1-963d-40d3-a98e-aab01d60f4db None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:22.838370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f66e1-963d-40d3-a98e-aab01d60f4db None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:22.839739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 509/1025] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:22.845165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08809a75-0486-480d-990f-2d70d9ef4f4f None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:22.845555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08809a75-0486-480d-990f-2d70d9ef4f4f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:22.845652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08809a75-0486-480d-990f-2d70d9ef4f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:22.862459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-08809a75-0486-480d-990f-2d70d9ef4f4f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:22.864479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-08809a75-0486-480d-990f-2d70d9ef4f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:22.891941 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c495-9827-40a7-8925-d17bf4fc5226 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:22.892268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c495-9827-40a7-8925-d17bf4fc5226 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:22.892552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4c495-9827-40a7-8925-d17bf4fc5226 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:22.894265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 510/1026] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:42:22.898969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2013e-55c0-4ddf-9e66-baae9c8b01d1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:22.899260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2013e-55c0-4ddf-9e66-baae9c8b01d1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:22.899539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2013e-55c0-4ddf-9e66-baae9c8b01d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:22.915969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-cbc2013e-55c0-4ddf-9e66-baae9c8b01d1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:22.917366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-cbc2013e-55c0-4ddf-9e66-baae9c8b01d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:22.944268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 517/1027] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:22.964655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08809a75-0486-480d-990f-2d70d9ef4f4f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:22.987596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 511/1028] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:23.011435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2013e-55c0-4ddf-9e66-baae9c8b01d1 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.050127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.059490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.071399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08809a75-0486-480d-990f-2d70d9ef4f4f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.107178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151d9bd-fe81-4f1d-a4be-d34554be4122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.108084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151d9bd-fe81-4f1d-a4be-d34554be4122 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.108404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151d9bd-fe81-4f1d-a4be-d34554be4122 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.108686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151d9bd-fe81-4f1d-a4be-d34554be4122 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.120572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2013e-55c0-4ddf-9e66-baae9c8b01d1 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.123674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151d9bd-fe81-4f1d-a4be-d34554be4122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-8151d9bd-fe81-4f1d-a4be-d34554be4122 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:23.134735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:23.135699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:23.136618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:23.137974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 518/1029] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 25 05:42:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:23.144182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151d9bd-fe81-4f1d-a4be-d34554be4122 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.158101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.159330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.159891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.160215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] PATH_INFO: `/v3/services/image` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.160973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.162404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:23.174413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:23.175976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:23.177208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 512/1030] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:23.182277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.258652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151d9bd-fe81-4f1d-a4be-d34554be4122 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.289770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d131de-ab8e-44d0-8ed8-763ffc7b8147 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.292515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.293263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.293550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.293825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.294571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:23.294879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.296267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.299346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.315010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 519/1031] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:23.315010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.315010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.331618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.332443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.332733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.334849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.334849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] RBAC: Adding query filter params (name=image) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:23.334849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.335527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.345074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 513/1032] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:23.346273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.412337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cf4d6-e7f5-404a-95df-fd1d138c9ba9 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.447229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.448019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.448325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.448597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.448859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b4c95-3c63-4221-b8b8-014390b7022c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.460190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.461631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:23.478790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:23.479729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:23.485570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:23.485570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:23.485570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:23.485570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:23.485570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 520/1033] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:23.485570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.490892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.491568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.491857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.492158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.492874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] RBAC: Adding query filter params (type=image) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:23.493184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.494555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.505262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 514/1034] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:23.506108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.542451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.552336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.599606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c422847-5b8e-4eb5-8277-f27cc2afd3e1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.616708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35ca03-0577-4dae-9f0f-ba69a490b069 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.636361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.637315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.637724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.638010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.638749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:23.639061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.640465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.652670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 521/1035] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:23.654900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.655824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.656346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.656823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.657824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] RBAC: Adding query filter params (interface=public, service_id=f6545161f1944acc824530b1cb8bf4fc, region_id=RegionOne) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:23.658091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.658656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.660279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.667398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 515/1036] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/endpoints?service_id=f6545161f1944acc824530b1cb8bf4fc&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:23.672886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.773473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc90b0-9d60-483f-96eb-8beef061774a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.780698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.784137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc7b87-8bce-4c0b-abb6-b91d8cd4d4d6 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.788940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:23.809345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.810138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.810426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.810709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] PATH_INFO: `/v3/users/nova` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.819378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.820363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31ba98-e5e1-4d97-a1fb-44b49ad15aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.820886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.821099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31ba98-e5e1-4d97-a1fb-44b49ad15aca None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.821388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31ba98-e5e1-4d97-a1fb-44b49ad15aca None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.821667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31ba98-e5e1-4d97-a1fb-44b49ad15aca None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.822352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31ba98-e5e1-4d97-a1fb-44b49ad15aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:23.823704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31ba98-e5e1-4d97-a1fb-44b49ad15aca None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:23.830372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 516/1037] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:42:23.837271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:23.838308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:23.839269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 522/1038] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:23.845374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31ba98-e5e1-4d97-a1fb-44b49ad15aca None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:23.957611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31ba98-e5e1-4d97-a1fb-44b49ad15aca None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:23.997158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a33133a-c68c-416c-94a8-f29f9b7c3cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:23.997846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33133a-c68c-416c-94a8-f29f9b7c3cc4 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:23.998131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33133a-c68c-416c-94a8-f29f9b7c3cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:23.998408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a33133a-c68c-416c-94a8-f29f9b7c3cc4 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:23.999249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33133a-c68c-416c-94a8-f29f9b7c3cc4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:23.999560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33133a-c68c-416c-94a8-f29f9b7c3cc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.001021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a33133a-c68c-416c-94a8-f29f9b7c3cc4 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.020674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 517/1039] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:42:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:24.026109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.093824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391b05c-e24b-4381-a78e-7c3ba38877ae None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.094152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391b05c-e24b-4381-a78e-7c3ba38877ae None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.094420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391b05c-e24b-4381-a78e-7c3ba38877ae None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.095801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 518/1040] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:24.101505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531891-f297-40b3-9b4d-7590a00e03bd None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.101799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531891-f297-40b3-9b4d-7590a00e03bd None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.102082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0531891-f297-40b3-9b4d-7590a00e03bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.119511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b0531891-f297-40b3-9b4d-7590a00e03bd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:24.120806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b0531891-f297-40b3-9b4d-7590a00e03bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:24.135904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6c40a-ba1b-4191-be55-82ab1b9fc94a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.139344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:24.177040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eac742-1070-4a48-b2ad-d552d5e770d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.177721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac742-1070-4a48-b2ad-d552d5e770d1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.178009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac742-1070-4a48-b2ad-d552d5e770d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.178288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eac742-1070-4a48-b2ad-d552d5e770d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.195342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 519/1041] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:24.219140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531891-f297-40b3-9b4d-7590a00e03bd None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.228254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:24.254312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eac742-1070-4a48-b2ad-d552d5e770d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.255757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eac742-1070-4a48-b2ad-d552d5e770d1 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.328517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0531891-f297-40b3-9b4d-7590a00e03bd None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.365393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446387c-5a85-46dc-a66f-ac5ec42a32d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.366191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446387c-5a85-46dc-a66f-ac5ec42a32d9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.366472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446387c-5a85-46dc-a66f-ac5ec42a32d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.366771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446387c-5a85-46dc-a66f-ac5ec42a32d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.367171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 523/1042] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:42:24.372755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eac742-1070-4a48-b2ad-d552d5e770d1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.377990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446387c-5a85-46dc-a66f-ac5ec42a32d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-3446387c-5a85-46dc-a66f-ac5ec42a32d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:24.385818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:24.386764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:24.387613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 520/1043] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:24.393010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446387c-5a85-46dc-a66f-ac5ec42a32d9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.409227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:24.456263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:24.482821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eac742-1070-4a48-b2ad-d552d5e770d1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.499252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446387c-5a85-46dc-a66f-ac5ec42a32d9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.521128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.521930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.522230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.522500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.533089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.534537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.537752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.538634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.539114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.539567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.540526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:24.541028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.542956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.553054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 521/1044] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:24.553291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:24.554168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:24.555060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:24.555060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:24.555060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:24.555060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:24.555060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 524/1045] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:24.563063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.563201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.640569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:24.667854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a13778-8155-4c31-8c3c-b3f791fd6f98 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.671135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322973ed-9042-4650-b94a-d055f9ff7912 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.698905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:24.707862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.708017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1cbb8-47a1-4cb6-8394-1d0bcd3c1a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.710194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.710304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1cbb8-47a1-4cb6-8394-1d0bcd3c1a53 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.710383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.710456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1cbb8-47a1-4cb6-8394-1d0bcd3c1a53 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.710526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.710610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1cbb8-47a1-4cb6-8394-1d0bcd3c1a53 None admin] PATH_INFO: `/v3/users/7aa1c4984fec4a6eb18dc0935349032b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.710684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:24.710684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.711480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.723390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 525/1046] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:24.725134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1cbb8-47a1-4cb6-8394-1d0bcd3c1a53 None admin] RBAC: Authorizing `identity:get_user(user_id=7aa1c4984fec4a6eb18dc0935349032b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.726594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1cbb8-47a1-4cb6-8394-1d0bcd3c1a53 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.732949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 522/1047] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/users/7aa1c4984fec4a6eb18dc0935349032b => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:24.733074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.733814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1cbb8-47a1-4cb6-8394-1d0bcd3c1a53 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.843735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a02d8-33ee-4ce7-a18a-94b25c49dcba None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.846258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1cbb8-47a1-4cb6-8394-1d0bcd3c1a53 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:24.878478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:24.882118 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.883172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.883570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664d35-22a2-46cb-87c8-d055133674e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:24.883929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.884318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664d35-22a2-46cb-87c8-d055133674e8 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:24.884611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664d35-22a2-46cb-87c8-d055133674e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:24.884817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.887544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91664d35-22a2-46cb-87c8-d055133674e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:24.892539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.894227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:42:24.909295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:24.910758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:24.911653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:42:24.911653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:42:24.911653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:24.911653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:24.911653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:24.911653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:24.911653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:24.911653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:24.918722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 526/1048] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:24.919785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:24.955932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91664d35-22a2-46cb-87c8-d055133674e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:24.958044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91664d35-22a2-46cb-87c8-d055133674e8 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:24.970839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:25.029501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616af89-5845-4fc6-babd-d454c85f572d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:25.067961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:25.068669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.069207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.069483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.070271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:25.070580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:25.071269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 523/1049] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:42:25.072020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:25.076862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664d35-22a2-46cb-87c8-d055133674e8 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:25.086410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 527/1050] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:42:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:25.092381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:25.109434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:25.186199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91664d35-22a2-46cb-87c8-d055133674e8 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:25.201030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741389d-3a42-40ba-be0a-7cf8c486f4fe None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:25.208685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:25.223386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf62a03-a932-477e-b7f6-998ed3c23ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:25.224286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf62a03-a932-477e-b7f6-998ed3c23ca2 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.230946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf62a03-a932-477e-b7f6-998ed3c23ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.230946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf62a03-a932-477e-b7f6-998ed3c23ca2 None admin] PATH_INFO: `/v3/projects/9301a34200504997ae9059147b385b53` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.230946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf62a03-a932-477e-b7f6-998ed3c23ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=9301a34200504997ae9059147b385b53)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:25.230946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf62a03-a932-477e-b7f6-998ed3c23ca2 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:25.230946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 524/1051] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:25 2022] GET /identity/v3/projects/9301a34200504997ae9059147b385b53 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:25.235326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf62a03-a932-477e-b7f6-998ed3c23ca2 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:25.236605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90952d6-9604-46e9-bca8-bb87e0289fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:25.237270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90952d6-9604-46e9-bca8-bb87e0289fb8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.237560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90952d6-9604-46e9-bca8-bb87e0289fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.238561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90952d6-9604-46e9-bca8-bb87e0289fb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.240229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90952d6-9604-46e9-bca8-bb87e0289fb8 None admin] RBAC: Adding query filter params (role.id=1c5a669f88fd44399a230e4f91a6c965, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=e6a0f84bf3f245bdab1391a2451cb4c0) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:25.240559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90952d6-9604-46e9-bca8-bb87e0289fb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:25.242002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90952d6-9604-46e9-bca8-bb87e0289fb8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:25.256199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 528/1052] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:25 2022] GET /identity/v3/role_assignments?user.id=e6a0f84bf3f245bdab1391a2451cb4c0&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=1c5a669f88fd44399a230e4f91a6c965 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:25.351066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:25.354437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf62a03-a932-477e-b7f6-998ed3c23ca2 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:25.390967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7bd96-d389-46a6-902c-11ccb2c1fcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:25.391640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7bd96-d389-46a6-902c-11ccb2c1fcb9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.391926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7bd96-d389-46a6-902c-11ccb2c1fcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.392235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7bd96-d389-46a6-902c-11ccb2c1fcb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.393079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7bd96-d389-46a6-902c-11ccb2c1fcb9 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=9301a34200504997ae9059147b385b53, user.id=7aa1c4984fec4a6eb18dc0935349032b) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:25.393393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7bd96-d389-46a6-902c-11ccb2c1fcb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:25.395535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7bd96-d389-46a6-902c-11ccb2c1fcb9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:25.403415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 525/1053] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:25 2022] GET /identity/v3/role_assignments?user.id=7aa1c4984fec4a6eb18dc0935349032b&scope.project.id=9301a34200504997ae9059147b385b53&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:25.924426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2d4c8-bd8a-4ee7-b273-0c87f79d1a8b None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.924719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2d4c8-bd8a-4ee7-b273-0c87f79d1a8b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.925202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2d4c8-bd8a-4ee7-b273-0c87f79d1a8b None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.926859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 529/1054] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:25.932240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef65cdb-498d-48fa-b538-03cc86808b05 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:25.932700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef65cdb-498d-48fa-b538-03cc86808b05 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:25.932806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef65cdb-498d-48fa-b538-03cc86808b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:25.949954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-aef65cdb-498d-48fa-b538-03cc86808b05 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:25.951135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-aef65cdb-498d-48fa-b538-03cc86808b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:26.019901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 526/1055] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:26.043302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2d4c8-bd8a-4ee7-b273-0c87f79d1a8b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.145328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2d4c8-bd8a-4ee7-b273-0c87f79d1a8b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.178850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.179638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.179919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.180218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] PATH_INFO: `/v3/services/image` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.180876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.182237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:26.194066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:26.195036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:26.201188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 530/1056] 10.222.0.40 () {56 vars in 1105 bytes} [Mon Jul 25 05:42:26 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:26.202011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef65cdb-498d-48fa-b538-03cc86808b05 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.292819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:26.299787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef65cdb-498d-48fa-b538-03cc86808b05 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.335229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.335904 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.336224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.336508 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.337220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] RBAC: Adding query filter params (name=image) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:26.337530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.338896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.348881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 527/1057] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:26.349677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.386427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:26.455686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f80b9-e274-482b-ab25-71888cb0c96e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.490402 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56608dc-0a1d-45b9-8887-fbe9e2340c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.491091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56608dc-0a1d-45b9-8887-fbe9e2340c0c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.491372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56608dc-0a1d-45b9-8887-fbe9e2340c0c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.491647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56608dc-0a1d-45b9-8887-fbe9e2340c0c None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.492403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56608dc-0a1d-45b9-8887-fbe9e2340c0c None admin] RBAC: Adding query filter params (type=image) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:26.492723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56608dc-0a1d-45b9-8887-fbe9e2340c0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.494112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56608dc-0a1d-45b9-8887-fbe9e2340c0c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.504888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 531/1058] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:26.506090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:26.604868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300f3b0-9632-40c1-b5f3-3140fa6f60b2 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:26.638735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26450e7-66df-46eb-9c6f-cf0b0474b19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:26.639621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26450e7-66df-46eb-9c6f-cf0b0474b19a None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:26.640152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26450e7-66df-46eb-9c6f-cf0b0474b19a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:26.640600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26450e7-66df-46eb-9c6f-cf0b0474b19a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:26.641549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26450e7-66df-46eb-9c6f-cf0b0474b19a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:26.643111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26450e7-66df-46eb-9c6f-cf0b0474b19a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:26.709601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 528/1059] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:42:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:42:27.307530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50bd2e-ac62-475c-92bd-d0d343ca3d56 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.308143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50bd2e-ac62-475c-92bd-d0d343ca3d56 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.308143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50bd2e-ac62-475c-92bd-d0d343ca3d56 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.309611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 532/1060] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:27.314682 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387839d-c29a-46f6-b059-7ac37d6f5f32 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.315093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387839d-c29a-46f6-b059-7ac37d6f5f32 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.315257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387839d-c29a-46f6-b059-7ac37d6f5f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.333384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e387839d-c29a-46f6-b059-7ac37d6f5f32 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:27.334567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e387839d-c29a-46f6-b059-7ac37d6f5f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:27.344139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:27.408371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 529/1061] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:27.430384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a632-e0e8-48f7-bf3d-08d06af08c9a None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.430707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a632-e0e8-48f7-bf3d-08d06af08c9a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.430953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803a632-e0e8-48f7-bf3d-08d06af08c9a None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.431224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387839d-c29a-46f6-b059-7ac37d6f5f32 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:27.432530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 533/1062] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:27.437279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077b384-c41d-430a-9e5a-8db2d8523229 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.437574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077b384-c41d-430a-9e5a-8db2d8523229 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.437859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077b384-c41d-430a-9e5a-8db2d8523229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.455115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-6077b384-c41d-430a-9e5a-8db2d8523229 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:27.456425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6077b384-c41d-430a-9e5a-8db2d8523229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:27.526098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 534/1063] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:27.533095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387839d-c29a-46f6-b059-7ac37d6f5f32 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:27.549131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077b384-c41d-430a-9e5a-8db2d8523229 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:27.570299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8e3f-9298-4259-a7fc-5f93e5cd6794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:27.571041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8e3f-9298-4259-a7fc-5f93e5cd6794 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.571330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8e3f-9298-4259-a7fc-5f93e5cd6794 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.571617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd8e3f-9298-4259-a7fc-5f93e5cd6794 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.577968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:27.579894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:27.583245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd8e3f-9298-4259-a7fc-5f93e5cd6794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b8cd8e3f-9298-4259-a7fc-5f93e5cd6794 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:27.591696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:27.592707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:27.594215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 530/1064] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 25 05:42:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:27.598908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8e3f-9298-4259-a7fc-5f93e5cd6794 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:27.659151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077b384-c41d-430a-9e5a-8db2d8523229 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:27.696315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b6bb5-3aa1-4b47-93f7-dd8c2ac7c4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:27.697120 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b6bb5-3aa1-4b47-93f7-dd8c2ac7c4cd None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.697400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b6bb5-3aa1-4b47-93f7-dd8c2ac7c4cd None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.697669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b6bb5-3aa1-4b47-93f7-dd8c2ac7c4cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.705401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd8e3f-9298-4259-a7fc-5f93e5cd6794 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:27.710320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b6bb5-3aa1-4b47-93f7-dd8c2ac7c4cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-3f7b6bb5-3aa1-4b47-93f7-dd8c2ac7c4cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:27.717987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:27.718935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:27.719786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 535/1065] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:27.725123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b6bb5-3aa1-4b47-93f7-dd8c2ac7c4cd None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:27.744596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:27.745267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.745553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.745832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.746584 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:27.746893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:27.748285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:27.762597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 531/1066] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 25 05:42:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:27.766284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:27.827966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:27.832655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:27.841260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b6bb5-3aa1-4b47-93f7-dd8c2ac7c4cd None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:27.875687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d76f65-a297-47a8-a0b0-f157155c3575 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:27.881157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:27.881842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.882127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.882407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.883162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:27.883477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:27.884893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:27.898447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 536/1067] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:27.899508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:27.917326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:27.918181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:27.918467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:27.918750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:27.930235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:27.931711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:27.948948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:27.950033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:27.950948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:27.950948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:27.950948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:27.950948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:27.950948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 532/1068] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:27.956752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.005542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c987c35-70e3-4876-96a7-2faaf9b6272f None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:28.041435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c968c7e-2276-4163-bfd7-5b17891dabef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:28.042314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c968c7e-2276-4163-bfd7-5b17891dabef None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.042599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c968c7e-2276-4163-bfd7-5b17891dabef None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.042885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c968c7e-2276-4163-bfd7-5b17891dabef None admin] PATH_INFO: `/v3/users/7aa1c4984fec4a6eb18dc0935349032b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.044085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c968c7e-2276-4163-bfd7-5b17891dabef None admin] RBAC: Authorizing `identity:get_user(user_id=7aa1c4984fec4a6eb18dc0935349032b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:28.045586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c968c7e-2276-4163-bfd7-5b17891dabef None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:28.047229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 537/1069] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:27 2022] GET /identity/v3/users/7aa1c4984fec4a6eb18dc0935349032b => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:28.053495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c968c7e-2276-4163-bfd7-5b17891dabef None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.065249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027c8ae-0614-494a-9ae5-76b8c3a20090 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:28.074328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:28.079326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:28.101386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:28.102058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.102358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.102612 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.103387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:28.103693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:28.105080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:28.121736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 533/1070] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:28.123199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.159029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c968c7e-2276-4163-bfd7-5b17891dabef None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:28.196912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651eafc-876d-4061-9d82-d2f37e5a2ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:28.197778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651eafc-876d-4061-9d82-d2f37e5a2ee2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.198059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651eafc-876d-4061-9d82-d2f37e5a2ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.198333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651eafc-876d-4061-9d82-d2f37e5a2ee2 None admin] PATH_INFO: `/v3/projects/9301a34200504997ae9059147b385b53` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.199520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651eafc-876d-4061-9d82-d2f37e5a2ee2 None admin] RBAC: Authorizing `identity:get_project(project_id=9301a34200504997ae9059147b385b53)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:28.200986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651eafc-876d-4061-9d82-d2f37e5a2ee2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:28.202951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 538/1071] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:28 2022] GET /identity/v3/projects/9301a34200504997ae9059147b385b53 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:28.209303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651eafc-876d-4061-9d82-d2f37e5a2ee2 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.232200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b953e0c-fcac-4ba2-ac0f-a9b5a611b9e5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:28.271321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:28.272518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.273067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.273514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.282115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:28.283789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:28.295850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:42:28.301690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:28.302851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:42:28.303833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:28.304416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 534/1072] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:42:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:28.309827 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.317783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651eafc-876d-4061-9d82-d2f37e5a2ee2 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:28.348351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:28.357809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35a0ca-a491-4672-843e-c2c5095a62e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:28.358712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35a0ca-a491-4672-843e-c2c5095a62e5 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.358996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35a0ca-a491-4672-843e-c2c5095a62e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.361244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35a0ca-a491-4672-843e-c2c5095a62e5 None admin] PATH_INFO: `/v3/projects/9301a34200504997ae9059147b385b53/users/7aa1c4984fec4a6eb18dc0935349032b/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.361382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35a0ca-a491-4672-843e-c2c5095a62e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9301a34200504997ae9059147b385b53, user_id=7aa1c4984fec4a6eb18dc0935349032b, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:28.363286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35a0ca-a491-4672-843e-c2c5095a62e5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:28.413298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 539/1073] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:42:28 2022] PUT /identity/v3/projects/9301a34200504997ae9059147b385b53/users/7aa1c4984fec4a6eb18dc0935349032b/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:28.421939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7a3f0-57be-456a-a75e-a6a2993686a9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:28.464587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:28.465300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.465589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.465870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.466790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:28.467108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:28.468635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:28.494257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 535/1074] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:42:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:28.495617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35a0ca-a491-4672-843e-c2c5095a62e5 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.608195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35a0ca-a491-4672-843e-c2c5095a62e5 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:28.643575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bd855-4c84-4723-a962-eb3f72cd3b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:28.644947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bd855-4c84-4723-a962-eb3f72cd3b90 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.645228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bd855-4c84-4723-a962-eb3f72cd3b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.645505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bd855-4c84-4723-a962-eb3f72cd3b90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.655727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bd855-4c84-4723-a962-eb3f72cd3b90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:28.657220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bd855-4c84-4723-a962-eb3f72cd3b90 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:28.665372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-a78bd855-4c84-4723-a962-eb3f72cd3b90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:28.677143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:28.682319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:28.683233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:28.683233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:28.683233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:28.683233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:28.683233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 540/1075] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:28.684938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.776183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198dc56-75d1-443d-b2ea-3cddc6945007 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.776475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198dc56-75d1-443d-b2ea-3cddc6945007 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.776760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198dc56-75d1-443d-b2ea-3cddc6945007 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.778154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 541/1076] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:42:28.783127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac2e17-c815-48ff-922e-7263b0e70398 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.783625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac2e17-c815-48ff-922e-7263b0e70398 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.784081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac2e17-c815-48ff-922e-7263b0e70398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.787661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:28.796800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1518bb-17a1-486d-92f3-e7159f2a4919 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:28.802134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-51ac2e17-c815-48ff-922e-7263b0e70398 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:28.803319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-51ac2e17-c815-48ff-922e-7263b0e70398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:28.834754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:28.835426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:28.835711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:28.835988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:28.836745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:28.837050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:28.838411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:28.850210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 536/1077] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:28.855993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.880204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 542/1078] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:28.901574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac2e17-c815-48ff-922e-7263b0e70398 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:28.966564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0293a-194b-4b5d-9ec5-36daa2876f17 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:29.008753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:29.009549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:29.009832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:29.010110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:29.011103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac2e17-c815-48ff-922e-7263b0e70398 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:29.016839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:29.018275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:42:29.032865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:29.033796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:29.034777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 537/1079] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:29.040328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:29.043260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:29.052198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7084b9-4927-4cfc-8b96-dca9efd09e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:29.052865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7084b9-4927-4cfc-8b96-dca9efd09e2d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:29.053150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7084b9-4927-4cfc-8b96-dca9efd09e2d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:29.053428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7084b9-4927-4cfc-8b96-dca9efd09e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:29.057900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:29.125259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7084b9-4927-4cfc-8b96-dca9efd09e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:29.126724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7084b9-4927-4cfc-8b96-dca9efd09e2d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:29.152666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b80c72-14ac-40cb-8067-e061e9849b39 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:29.190245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:29.191472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:29.191761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:29.192039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:29.192868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:29.193183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:29.194884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:29.208150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 538/1080] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:29.212568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:29.240560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 543/1081] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:42:29.246195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7084b9-4927-4cfc-8b96-dca9efd09e2d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:29.274547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:29.290115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:29.321151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5de44-1413-4ab5-ae94-49a151594662 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:29.350040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7084b9-4927-4cfc-8b96-dca9efd09e2d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:29.356270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f82a22-f866-49ac-9d5a-bc620829c81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:29.357164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f82a22-f866-49ac-9d5a-bc620829c81c None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:29.357443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f82a22-f866-49ac-9d5a-bc620829c81c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:29.357723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f82a22-f866-49ac-9d5a-bc620829c81c None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/e6a0f84bf3f245bdab1391a2451cb4c0/roles/1c5a669f88fd44399a230e4f91a6c965` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:29.359901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f82a22-f866-49ac-9d5a-bc620829c81c None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=e6a0f84bf3f245bdab1391a2451cb4c0, role_id=1c5a669f88fd44399a230e4f91a6c965)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:29.361360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f82a22-f866-49ac-9d5a-bc620829c81c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:29.385276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:29.386061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:29.386363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:29.386615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:29.397291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:29.398717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:29.416701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:29.423581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:29.424517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:29.424517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:29.424517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:29.424517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:29.424517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 544/1082] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:29.424517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:29.441562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 539/1083] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:42:29 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/e6a0f84bf3f245bdab1391a2451cb4c0/roles/1c5a669f88fd44399a230e4f91a6c965 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:42:29.515014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:29.537765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b058c77-6724-4a9f-91dc-e0184fd8da17 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:29.577536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327aeaf4-6dc0-4e6d-b840-c237528541d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:29.578228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aeaf4-6dc0-4e6d-b840-c237528541d3 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:29.578513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aeaf4-6dc0-4e6d-b840-c237528541d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:29.578793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327aeaf4-6dc0-4e6d-b840-c237528541d3 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:29.579543 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327aeaf4-6dc0-4e6d-b840-c237528541d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:29.579852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327aeaf4-6dc0-4e6d-b840-c237528541d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:29.581407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327aeaf4-6dc0-4e6d-b840-c237528541d3 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:29.600139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 545/1084] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:30.474144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae6e14-7c53-49f4-ace3-c709f02453b5 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:30.474685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae6e14-7c53-49f4-ace3-c709f02453b5 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:30.474685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae6e14-7c53-49f4-ace3-c709f02453b5 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:30.476354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 540/1085] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:30.481319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497a85e-46db-424f-8c2c-7378ac532356 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:30.481721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497a85e-46db-424f-8c2c-7378ac532356 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:30.481879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497a85e-46db-424f-8c2c-7378ac532356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:30.498259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d497a85e-46db-424f-8c2c-7378ac532356 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:30.499437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d497a85e-46db-424f-8c2c-7378ac532356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:30.566600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 546/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:30.590549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae6e14-7c53-49f4-ace3-c709f02453b5 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:30.690948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae6e14-7c53-49f4-ace3-c709f02453b5 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:30.719494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:30.724898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de5331-7349-4438-828e-dfd8e349e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:30.725627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de5331-7349-4438-828e-dfd8e349e1a0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:30.725910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de5331-7349-4438-828e-dfd8e349e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:30.726186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de5331-7349-4438-828e-dfd8e349e1a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:30.735850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de5331-7349-4438-828e-dfd8e349e1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-54de5331-7349-4438-828e-dfd8e349e1a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:30.743435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:30.744477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:30.745310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 541/1087] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:30.752153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497a85e-46db-424f-8c2c-7378ac532356 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:30.819279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:30.849122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497a85e-46db-424f-8c2c-7378ac532356 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:30.882837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:30.883503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:30.883785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:30.884056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:30.884832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:30.885134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:30.886514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:30.895109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 547/1088] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:30.900922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de5331-7349-4438-828e-dfd8e349e1a0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:30.998239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de5331-7349-4438-828e-dfd8e349e1a0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:31.031116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59569d0e-4606-4f6f-85da-5042bd508346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:31.031973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59569d0e-4606-4f6f-85da-5042bd508346 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.032287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59569d0e-4606-4f6f-85da-5042bd508346 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.032565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59569d0e-4606-4f6f-85da-5042bd508346 None admin] PATH_INFO: `/v3/users/7aa1c4984fec4a6eb18dc0935349032b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.033728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59569d0e-4606-4f6f-85da-5042bd508346 None admin] RBAC: Authorizing `identity:get_user(user_id=7aa1c4984fec4a6eb18dc0935349032b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:31.035329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59569d0e-4606-4f6f-85da-5042bd508346 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:31.037214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 542/1089] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:30 2022] GET /identity/v3/users/7aa1c4984fec4a6eb18dc0935349032b => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:31.042722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:31.140315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c199d-ebab-4965-812c-618b98f2252b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:31.170142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:31.175010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac66bfd-a544-411f-9db9-85284a520f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:31.175808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac66bfd-a544-411f-9db9-85284a520f86 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.176099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac66bfd-a544-411f-9db9-85284a520f86 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.176412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac66bfd-a544-411f-9db9-85284a520f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.243406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac66bfd-a544-411f-9db9-85284a520f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:31.244905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac66bfd-a544-411f-9db9-85284a520f86 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:31.341354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 548/1090] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:31.347716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59569d0e-4606-4f6f-85da-5042bd508346 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:31.405533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:31.445927 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59569d0e-4606-4f6f-85da-5042bd508346 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:31.479744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ef7da7-8725-469f-a3b0-e8b05e9d001e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:31.480638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef7da7-8725-469f-a3b0-e8b05e9d001e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.480921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef7da7-8725-469f-a3b0-e8b05e9d001e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.481195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ef7da7-8725-469f-a3b0-e8b05e9d001e None admin] PATH_INFO: `/v3/projects/9301a34200504997ae9059147b385b53` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.482374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef7da7-8725-469f-a3b0-e8b05e9d001e None admin] RBAC: Authorizing `identity:get_project(project_id=9301a34200504997ae9059147b385b53)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:31.483814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ef7da7-8725-469f-a3b0-e8b05e9d001e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:31.485542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 543/1091] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:31 2022] GET /identity/v3/projects/9301a34200504997ae9059147b385b53 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:31.491630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac66bfd-a544-411f-9db9-85284a520f86 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:31.501883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5172d1-6941-4805-b171-7a953c867923 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.502165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5172d1-6941-4805-b171-7a953c867923 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.502443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5172d1-6941-4805-b171-7a953c867923 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.504430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 544/1092] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:42:31.509260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77ed9-0de1-4e45-8a0b-0619ed7f53e5 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.509546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77ed9-0de1-4e45-8a0b-0619ed7f53e5 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.509824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c77ed9-0de1-4e45-8a0b-0619ed7f53e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.523119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:31.526859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-46c77ed9-0de1-4e45-8a0b-0619ed7f53e5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:31.528038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-46c77ed9-0de1-4e45-8a0b-0619ed7f53e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:31.593337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac66bfd-a544-411f-9db9-85284a520f86 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:31.594862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 545/1093] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:31.618546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c77ed9-0de1-4e45-8a0b-0619ed7f53e5 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:31.626276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:31.626937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.627215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.627489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.629103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=9301a34200504997ae9059147b385b53, user.id=7aa1c4984fec4a6eb18dc0935349032b) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:31.629480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:31.630909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:31.648142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 549/1094] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:31 2022] GET /identity/v3/role_assignments?user.id=7aa1c4984fec4a6eb18dc0935349032b&scope.project.id=9301a34200504997ae9059147b385b53&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:31.680045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:31.727628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c77ed9-0de1-4e45-8a0b-0619ed7f53e5 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:31.768162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cdecc-b272-436c-9ae1-6b5016620241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:31.768162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cdecc-b272-436c-9ae1-6b5016620241 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.768162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cdecc-b272-436c-9ae1-6b5016620241 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.768162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cdecc-b272-436c-9ae1-6b5016620241 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.780945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cdecc-b272-436c-9ae1-6b5016620241 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-af6cdecc-b272-436c-9ae1-6b5016620241 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:31.789408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:31.790543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 25 05:42:31.791589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:31.792507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 546/1095] 10.222.0.40 () {56 vars in 1120 bytes} [Mon Jul 25 05:42:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:31.798102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:31.903748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:31.910312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82278f9b-2d69-4ca8-8089-ca54d38f6e7c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:31.947309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:31.947985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:31.948300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:31.948575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:31.949322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:31.949627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:31.951003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:31.960096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 550/1096] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 25 05:42:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:31.965446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cdecc-b272-436c-9ae1-6b5016620241 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:32.070283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cdecc-b272-436c-9ae1-6b5016620241 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:32.087241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:32.104220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:32.105010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:32.105296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:32.105570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:32.117030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:32.118490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:32.135840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:32.136794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:32.137696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:32.137696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:32.137696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:32.137696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:32.137696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 547/1097] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:32.142844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:32.240423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05bb250-b893-4c4f-b4ee-0ebe8e00030b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:32.273193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:32.273853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:32.274135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:32.274410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:32.275130 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:32.275438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:32.276838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:32.288715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 551/1098] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:32.294322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:32.391927 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf71cf-0274-4449-88b0-6ef67444c6c7 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:32.425990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:32.426780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:32.427064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:32.427339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] PATH_INFO: `/v3/users/nova` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:32.434673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:32.436099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:42:32.450174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:32.451194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:32.452554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 548/1099] 10.222.0.40 () {56 vars in 1093 bytes} [Mon Jul 25 05:42:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:32.457204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:32.476235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:32.554154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b339a0c-fc2b-483c-aca8-bab9c63eaf40 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:32.587014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:32.587676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:32.587957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:32.588275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:32.589106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:32.589413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:32.590832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:32.607708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 552/1100] 10.222.0.40 () {56 vars in 1133 bytes} [Mon Jul 25 05:42:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:32.613984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:32.644600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:32.714151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24e93b-db91-419a-ac7c-9b2bcfbf711a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:32.747746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932b35b-4aea-4063-9217-ae132f9d7624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:32.748433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932b35b-4aea-4063-9217-ae132f9d7624 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:32.748718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932b35b-4aea-4063-9217-ae132f9d7624 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:32.749002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932b35b-4aea-4063-9217-ae132f9d7624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:32.820588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932b35b-4aea-4063-9217-ae132f9d7624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:32.822033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932b35b-4aea-4063-9217-ae132f9d7624 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:32.851519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:32.919602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 549/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:32.926049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:33.007385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:33.029491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743efa39-8efc-4f32-b143-d008e71c3922 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:33.063231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:33.064017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.064331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.064613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.074783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:33.076191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:42:33.093364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:33.094416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:33.095506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:42:33.095506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:42:33.095506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:42:33.095506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:33.096267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 553/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:33.100724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932b35b-4aea-4063-9217-ae132f9d7624 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:33.199114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932b35b-4aea-4063-9217-ae132f9d7624 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:33.226777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:33.232523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:33.233186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.233470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.233748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.234466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:33.234773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:33.236156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:33.248281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 550/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:33.254150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:33.353726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718382f4-9d9f-48ba-8017-6953aa9e8c3b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:33.379864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:33.387360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:33.391378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.391378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.391378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.396095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:33.397565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:42:33.412660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:33.413562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:33.414457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 554/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:42:33.421111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:33.518101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd856286-3205-484d-947a-20eb4d1e4218 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:33.551239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460080ce-46b7-4276-960a-4f97c07cc683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:33.551895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460080ce-46b7-4276-960a-4f97c07cc683 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.552209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460080ce-46b7-4276-960a-4f97c07cc683 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.552490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460080ce-46b7-4276-960a-4f97c07cc683 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.553268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460080ce-46b7-4276-960a-4f97c07cc683 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:33.553577 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460080ce-46b7-4276-960a-4f97c07cc683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:33.554992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460080ce-46b7-4276-960a-4f97c07cc683 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:33.567056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 551/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:33.573708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:33.674984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ffe10-6a6b-4d59-8bd9-f8f84a9042db None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:33.709469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:33.710132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.710414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.710688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.711521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] RBAC: Adding query filter params (role.id=1c5a669f88fd44399a230e4f91a6c965, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=e6a0f84bf3f245bdab1391a2451cb4c0) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:33.711834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:33.713308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:33.715790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cfc9-d5ee-47f0-ac54-01f9079eed0e None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.716254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cfc9-d5ee-47f0-ac54-01f9079eed0e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.716387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31cfc9-d5ee-47f0-ac54-01f9079eed0e None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.717887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 552/1106] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:42:33.719771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 555/1107] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:33 2022] GET /identity/v3/role_assignments?user.id=e6a0f84bf3f245bdab1391a2451cb4c0&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=1c5a669f88fd44399a230e4f91a6c965 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:33.723261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81185b-7492-4cc1-89c7-934b6c0940dd None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.723545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81185b-7492-4cc1-89c7-934b6c0940dd None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.723822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a81185b-7492-4cc1-89c7-934b6c0940dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.741952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2a81185b-7492-4cc1-89c7-934b6c0940dd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:33.743175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2a81185b-7492-4cc1-89c7-934b6c0940dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:33.795645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:33.817547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 553/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:33.843180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:33.899861 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:33.948178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebebe-91fa-4428-b7a4-e6a6dd32c753 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:33.986630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c03665-3337-4c27-b14d-b173a450fe94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:33.987863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c03665-3337-4c27-b14d-b173a450fe94 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:33.988210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c03665-3337-4c27-b14d-b173a450fe94 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:33.988437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c03665-3337-4c27-b14d-b173a450fe94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:33.989620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c03665-3337-4c27-b14d-b173a450fe94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:33.991053 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c03665-3337-4c27-b14d-b173a450fe94 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:33.992719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 556/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:33.998393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81185b-7492-4cc1-89c7-934b6c0940dd None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:34.101976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a81185b-7492-4cc1-89c7-934b6c0940dd None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:34.138667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9ec9b-8a3f-494c-89f7-4b453cbadadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:34.138667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ec9b-8a3f-494c-89f7-4b453cbadadd None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:34.138958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ec9b-8a3f-494c-89f7-4b453cbadadd None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:34.139187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9ec9b-8a3f-494c-89f7-4b453cbadadd None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:34.139894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9ec9b-8a3f-494c-89f7-4b453cbadadd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:34.141352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9ec9b-8a3f-494c-89f7-4b453cbadadd None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:34.219939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 554/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:42:33 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:36.296679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695fb6a-1435-438d-a9a3-9065362a2934 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:36.297212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695fb6a-1435-438d-a9a3-9065362a2934 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:36.297300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695fb6a-1435-438d-a9a3-9065362a2934 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:36.298613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 557/1111] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:36.303972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b2429-7da5-4c7e-95a6-5356c03d3b15 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:36.304427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b2429-7da5-4c7e-95a6-5356c03d3b15 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:36.304561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b2429-7da5-4c7e-95a6-5356c03d3b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:36.321597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a13b2429-7da5-4c7e-95a6-5356c03d3b15 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:36.322765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a13b2429-7da5-4c7e-95a6-5356c03d3b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:36.351077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:36.387992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 555/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:36.412065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695fb6a-1435-438d-a9a3-9065362a2934 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:36.512098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695fb6a-1435-438d-a9a3-9065362a2934 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:36.531555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:36.544936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0994e4-9d45-4c1f-90f0-e385b33b8d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:36.545785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0994e4-9d45-4c1f-90f0-e385b33b8d3a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:36.546070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0994e4-9d45-4c1f-90f0-e385b33b8d3a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:36.546347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0994e4-9d45-4c1f-90f0-e385b33b8d3a None admin] PATH_INFO: `/v3/roles/20a552366bf14792b05decae218a79ee` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:36.547924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0994e4-9d45-4c1f-90f0-e385b33b8d3a None admin] RBAC: Authorizing `identity:get_role(role_id=20a552366bf14792b05decae218a79ee)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:36.549299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0994e4-9d45-4c1f-90f0-e385b33b8d3a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:36.551156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 558/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:36 2022] GET /identity/v3/roles/20a552366bf14792b05decae218a79ee => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:36.556966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b2429-7da5-4c7e-95a6-5356c03d3b15 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:36.651724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b2429-7da5-4c7e-95a6-5356c03d3b15 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:36.684203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338d03d-b2b2-44ac-990b-9690810e7215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:36.685058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338d03d-b2b2-44ac-990b-9690810e7215 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:36.685338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338d03d-b2b2-44ac-990b-9690810e7215 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:36.685611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338d03d-b2b2-44ac-990b-9690810e7215 None admin] PATH_INFO: `/v3/users/5e7b97b4a53e4f6cb4016620b28da1eb` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:36.699590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338d03d-b2b2-44ac-990b-9690810e7215 None admin] RBAC: Authorizing `identity:get_user(user_id=5e7b97b4a53e4f6cb4016620b28da1eb)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:36.701049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2338d03d-b2b2-44ac-990b-9690810e7215 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:36.702650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 556/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:36 2022] GET /identity/v3/users/5e7b97b4a53e4f6cb4016620b28da1eb => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:36.708523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0994e4-9d45-4c1f-90f0-e385b33b8d3a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:36.805105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0994e4-9d45-4c1f-90f0-e385b33b8d3a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:36.837337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4d007-4a58-40d3-a441-88d417ae24c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:36.837985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d007-4a58-40d3-a441-88d417ae24c2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:36.838268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d007-4a58-40d3-a441-88d417ae24c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:36.838545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4d007-4a58-40d3-a441-88d417ae24c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:36.892472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:36.904353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4d007-4a58-40d3-a441-88d417ae24c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:36.905796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d4d007-4a58-40d3-a441-88d417ae24c2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:37.001901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 559/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:37.008302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338d03d-b2b2-44ac-990b-9690810e7215 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:37.033566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:37.107197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338d03d-b2b2-44ac-990b-9690810e7215 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:37.140017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b1ffb-dcba-4e77-8164-34289c11d54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:37.140894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b1ffb-dcba-4e77-8164-34289c11d54f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:37.141177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b1ffb-dcba-4e77-8164-34289c11d54f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:37.141451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b1ffb-dcba-4e77-8164-34289c11d54f None admin] PATH_INFO: `/v3/projects/9301a34200504997ae9059147b385b53` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:37.142648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b1ffb-dcba-4e77-8164-34289c11d54f None admin] RBAC: Authorizing `identity:get_project(project_id=9301a34200504997ae9059147b385b53)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:37.144700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b1ffb-dcba-4e77-8164-34289c11d54f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:37.146416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 557/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:37 2022] GET /identity/v3/projects/9301a34200504997ae9059147b385b53 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:37.152833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4d007-4a58-40d3-a441-88d417ae24c2 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:37.237046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:37.248491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d4d007-4a58-40d3-a441-88d417ae24c2 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:37.281391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b8f27-7acb-4859-8ab2-5759525cdac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:37.282044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8f27-7acb-4859-8ab2-5759525cdac5 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:37.282321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8f27-7acb-4859-8ab2-5759525cdac5 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:37.282590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b8f27-7acb-4859-8ab2-5759525cdac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:37.283414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b8f27-7acb-4859-8ab2-5759525cdac5 None admin] RBAC: Adding query filter params (role.id=20a552366bf14792b05decae218a79ee, scope.project.id=9301a34200504997ae9059147b385b53, user.id=5e7b97b4a53e4f6cb4016620b28da1eb) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:37.283722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b8f27-7acb-4859-8ab2-5759525cdac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:37.286026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b8f27-7acb-4859-8ab2-5759525cdac5 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:37.291258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 560/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:37 2022] GET /identity/v3/role_assignments?user.id=5e7b97b4a53e4f6cb4016620b28da1eb&scope.project.id=9301a34200504997ae9059147b385b53&role.id=20a552366bf14792b05decae218a79ee => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:39.310061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99278ab8-7179-425d-af2e-a06a45318c59 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:39.310613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99278ab8-7179-425d-af2e-a06a45318c59 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:39.310613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99278ab8-7179-425d-af2e-a06a45318c59 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:39.311978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 558/1118] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:39.317198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b814-0f74-446b-8380-a654a0926775 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:39.317638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b814-0f74-446b-8380-a654a0926775 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:39.317743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1b814-0f74-446b-8380-a654a0926775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:39.334340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-09b1b814-0f74-446b-8380-a654a0926775 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:39.335525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-09b1b814-0f74-446b-8380-a654a0926775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:39.400507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 561/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:39.423663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99278ab8-7179-425d-af2e-a06a45318c59 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:39.512357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:39.523209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99278ab8-7179-425d-af2e-a06a45318c59 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:39.559258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cce3ec-88f9-438f-a7e6-eec3c313b2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:39.560244 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cce3ec-88f9-438f-a7e6-eec3c313b2c9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:39.560477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cce3ec-88f9-438f-a7e6-eec3c313b2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:39.560751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cce3ec-88f9-438f-a7e6-eec3c313b2c9 None admin] PATH_INFO: `/v3/roles/20a552366bf14792b05decae218a79ee` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:39.562488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cce3ec-88f9-438f-a7e6-eec3c313b2c9 None admin] RBAC: Authorizing `identity:get_role(role_id=20a552366bf14792b05decae218a79ee)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:39.563979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cce3ec-88f9-438f-a7e6-eec3c313b2c9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:39.565874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 559/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:39 2022] GET /identity/v3/roles/20a552366bf14792b05decae218a79ee => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:39.572686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1b814-0f74-446b-8380-a654a0926775 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:39.652561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:39.670109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1b814-0f74-446b-8380-a654a0926775 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:39.702884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63db33-7e0a-49d5-b538-ee97e4471d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:39.703750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63db33-7e0a-49d5-b538-ee97e4471d4a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:39.704032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63db33-7e0a-49d5-b538-ee97e4471d4a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:39.704336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63db33-7e0a-49d5-b538-ee97e4471d4a None admin] PATH_INFO: `/v3/users/5e7b97b4a53e4f6cb4016620b28da1eb` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:39.706077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63db33-7e0a-49d5-b538-ee97e4471d4a None admin] RBAC: Authorizing `identity:get_user(user_id=5e7b97b4a53e4f6cb4016620b28da1eb)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:39.707549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c63db33-7e0a-49d5-b538-ee97e4471d4a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:39.709197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 562/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:39 2022] GET /identity/v3/users/5e7b97b4a53e4f6cb4016620b28da1eb => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:39.715159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cce3ec-88f9-438f-a7e6-eec3c313b2c9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:39.813400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cce3ec-88f9-438f-a7e6-eec3c313b2c9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:39.847255 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a325c-5743-4f86-8199-ab213a0929a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:39.848105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a325c-5743-4f86-8199-ab213a0929a3 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:39.848421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a325c-5743-4f86-8199-ab213a0929a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:39.848699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a325c-5743-4f86-8199-ab213a0929a3 None admin] PATH_INFO: `/v3/projects/9301a34200504997ae9059147b385b53` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:39.849906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a325c-5743-4f86-8199-ab213a0929a3 None admin] RBAC: Authorizing `identity:get_project(project_id=9301a34200504997ae9059147b385b53)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:39.851334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2a325c-5743-4f86-8199-ab213a0929a3 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:39.853057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 560/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:39 2022] GET /identity/v3/projects/9301a34200504997ae9059147b385b53 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:39.859177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63db33-7e0a-49d5-b538-ee97e4471d4a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:39.956326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63db33-7e0a-49d5-b538-ee97e4471d4a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:39.988689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df2361-0d43-4928-b2b4-8f4df77617bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:39.989559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2361-0d43-4928-b2b4-8f4df77617bc None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:39.989841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2361-0d43-4928-b2b4-8f4df77617bc None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:39.990115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df2361-0d43-4928-b2b4-8f4df77617bc None admin] PATH_INFO: `/v3/projects/9301a34200504997ae9059147b385b53/users/5e7b97b4a53e4f6cb4016620b28da1eb/roles/20a552366bf14792b05decae218a79ee` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:39.992305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df2361-0d43-4928-b2b4-8f4df77617bc None admin] RBAC: Authorizing `identity:create_grant(project_id=9301a34200504997ae9059147b385b53, user_id=5e7b97b4a53e4f6cb4016620b28da1eb, role_id=20a552366bf14792b05decae218a79ee)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:39.993725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df2361-0d43-4928-b2b4-8f4df77617bc None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:40.090725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 563/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:42:39 2022] PUT /identity/v3/projects/9301a34200504997ae9059147b385b53/users/5e7b97b4a53e4f6cb4016620b28da1eb/roles/20a552366bf14792b05decae218a79ee => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:42.106830 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe080db-f3d8-499d-8cf0-cb5a274b787d None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:42.107363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe080db-f3d8-499d-8cf0-cb5a274b787d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:42.107363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe080db-f3d8-499d-8cf0-cb5a274b787d None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:42.108742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 561/1124] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:42.114798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99aeb8-c275-4b5d-968e-ef53c34b06da None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:42.115192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99aeb8-c275-4b5d-968e-ef53c34b06da None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:42.115344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99aeb8-c275-4b5d-968e-ef53c34b06da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:42.131614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-7a99aeb8-c275-4b5d-968e-ef53c34b06da None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:42.132815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7a99aeb8-c275-4b5d-968e-ef53c34b06da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:42.135526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:42.199134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 564/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:42.223239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe080db-f3d8-499d-8cf0-cb5a274b787d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:42.250982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:42.321809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe080db-f3d8-499d-8cf0-cb5a274b787d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:42.354241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32935f7-1ee6-4050-9674-8fe7b7fb7773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:42.355087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32935f7-1ee6-4050-9674-8fe7b7fb7773 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:42.355367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32935f7-1ee6-4050-9674-8fe7b7fb7773 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:42.355638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32935f7-1ee6-4050-9674-8fe7b7fb7773 None admin] PATH_INFO: `/v3/roles/20a552366bf14792b05decae218a79ee` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:42.357162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32935f7-1ee6-4050-9674-8fe7b7fb7773 None admin] RBAC: Authorizing `identity:get_role(role_id=20a552366bf14792b05decae218a79ee)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:42.358463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32935f7-1ee6-4050-9674-8fe7b7fb7773 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:42.359802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 562/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:42 2022] GET /identity/v3/roles/20a552366bf14792b05decae218a79ee => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:42.366603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99aeb8-c275-4b5d-968e-ef53c34b06da None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:42.463709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99aeb8-c275-4b5d-968e-ef53c34b06da None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:42.496944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fabd5-9bde-4c14-a0dd-f243d6e395e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:42.497789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fabd5-9bde-4c14-a0dd-f243d6e395e3 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:42.498066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fabd5-9bde-4c14-a0dd-f243d6e395e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:42.498336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fabd5-9bde-4c14-a0dd-f243d6e395e3 None admin] PATH_INFO: `/v3/users/5e7b97b4a53e4f6cb4016620b28da1eb` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:42.499490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fabd5-9bde-4c14-a0dd-f243d6e395e3 None admin] RBAC: Authorizing `identity:get_user(user_id=5e7b97b4a53e4f6cb4016620b28da1eb)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:42.501006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fabd5-9bde-4c14-a0dd-f243d6e395e3 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:42.503396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 565/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:42 2022] GET /identity/v3/users/5e7b97b4a53e4f6cb4016620b28da1eb => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:42.508987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32935f7-1ee6-4050-9674-8fe7b7fb7773 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:42.600243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:42.605729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32935f7-1ee6-4050-9674-8fe7b7fb7773 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:42.637148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf63f70-17d4-4808-9766-4ac841b7c1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:42.637718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63f70-17d4-4808-9766-4ac841b7c1ce None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:42.638005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63f70-17d4-4808-9766-4ac841b7c1ce None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:42.638280 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63f70-17d4-4808-9766-4ac841b7c1ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:42.702084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf63f70-17d4-4808-9766-4ac841b7c1ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:42.703528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf63f70-17d4-4808-9766-4ac841b7c1ce None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:42.792074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:42.798274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 563/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:42.804354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fabd5-9bde-4c14-a0dd-f243d6e395e3 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:42.815656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:42.900883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fabd5-9bde-4c14-a0dd-f243d6e395e3 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:42.932981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dc60bd-4c62-4492-8274-f034bd217713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:42.933829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc60bd-4c62-4492-8274-f034bd217713 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:42.934109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc60bd-4c62-4492-8274-f034bd217713 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:42.934384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dc60bd-4c62-4492-8274-f034bd217713 None admin] PATH_INFO: `/v3/projects/9301a34200504997ae9059147b385b53` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:42.935532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc60bd-4c62-4492-8274-f034bd217713 None admin] RBAC: Authorizing `identity:get_project(project_id=9301a34200504997ae9059147b385b53)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:42.937055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dc60bd-4c62-4492-8274-f034bd217713 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:42.938738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 566/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:42 2022] GET /identity/v3/projects/9301a34200504997ae9059147b385b53 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:42.945556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf63f70-17d4-4808-9766-4ac841b7c1ce None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:43.041412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf63f70-17d4-4808-9766-4ac841b7c1ce None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:43.074825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb867a-bf3b-4100-b5b9-bf1c7962b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:43.075455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb867a-bf3b-4100-b5b9-bf1c7962b30c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:43.075734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb867a-bf3b-4100-b5b9-bf1c7962b30c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:43.076006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb867a-bf3b-4100-b5b9-bf1c7962b30c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:43.076851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb867a-bf3b-4100-b5b9-bf1c7962b30c None admin] RBAC: Adding query filter params (role.id=20a552366bf14792b05decae218a79ee, scope.project.id=9301a34200504997ae9059147b385b53, user.id=5e7b97b4a53e4f6cb4016620b28da1eb) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:43.077156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb867a-bf3b-4100-b5b9-bf1c7962b30c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:43.078562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb867a-bf3b-4100-b5b9-bf1c7962b30c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:43.085141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 564/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:42 2022] GET /identity/v3/role_assignments?user.id=5e7b97b4a53e4f6cb4016620b28da1eb&scope.project.id=9301a34200504997ae9059147b385b53&role.id=20a552366bf14792b05decae218a79ee => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:45.131435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adae06-d3f6-4b16-9b3d-9b5595867a20 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:45.131941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adae06-d3f6-4b16-9b3d-9b5595867a20 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:45.131941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85adae06-d3f6-4b16-9b3d-9b5595867a20 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:45.133294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 567/1131] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:45.138393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701dbf-fe1b-4ab2-9c2c-6a9d976134c3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:45.138828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701dbf-fe1b-4ab2-9c2c-6a9d976134c3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:45.138955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5701dbf-fe1b-4ab2-9c2c-6a9d976134c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:45.155422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a5701dbf-fe1b-4ab2-9c2c-6a9d976134c3 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:45.156658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a5701dbf-fe1b-4ab2-9c2c-6a9d976134c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:45.210929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:45.222319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 565/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:45.245406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adae06-d3f6-4b16-9b3d-9b5595867a20 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:45.320305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:45.342945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85adae06-d3f6-4b16-9b3d-9b5595867a20 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:45.376224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad4a95-fc36-47f3-9e62-0893b01649e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:45.377018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad4a95-fc36-47f3-9e62-0893b01649e2 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:45.377718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad4a95-fc36-47f3-9e62-0893b01649e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:45.377996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad4a95-fc36-47f3-9e62-0893b01649e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:45.379164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad4a95-fc36-47f3-9e62-0893b01649e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:45.380621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad4a95-fc36-47f3-9e62-0893b01649e2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:45.382276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 568/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:45.387907 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5701dbf-fe1b-4ab2-9c2c-6a9d976134c3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:45.487399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5701dbf-fe1b-4ab2-9c2c-6a9d976134c3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:45.519792 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013c077-598a-4c8a-b433-9a711ee982a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:45.520468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013c077-598a-4c8a-b433-9a711ee982a0 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:45.520755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013c077-598a-4c8a-b433-9a711ee982a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:45.521034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013c077-598a-4c8a-b433-9a711ee982a0 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:45.521742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013c077-598a-4c8a-b433-9a711ee982a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:45.523160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013c077-598a-4c8a-b433-9a711ee982a0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:45.611932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 566/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:42:45 2022] POST /identity/v3/projects => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:47.666545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54545658-01aa-4255-acba-7885cde924fa None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:47.667109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54545658-01aa-4255-acba-7885cde924fa None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:47.667109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54545658-01aa-4255-acba-7885cde924fa None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:47.668514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 569/1135] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:47.673674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f45b3-5631-4af5-b331-d38ec6b587b0 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:47.674094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f45b3-5631-4af5-b331-d38ec6b587b0 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:47.674745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f45b3-5631-4af5-b331-d38ec6b587b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:47.691928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8f6f45b3-5631-4af5-b331-d38ec6b587b0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:47.693407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8f6f45b3-5631-4af5-b331-d38ec6b587b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:47.727207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:47.758500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 567/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:47.782321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54545658-01aa-4255-acba-7885cde924fa None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:47.882837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54545658-01aa-4255-acba-7885cde924fa None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:47.915529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab11e06-8741-4f5d-a8e1-6c6b42200d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:47.916341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab11e06-8741-4f5d-a8e1-6c6b42200d7d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:47.916677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab11e06-8741-4f5d-a8e1-6c6b42200d7d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:47.916911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab11e06-8741-4f5d-a8e1-6c6b42200d7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:47.918102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab11e06-8741-4f5d-a8e1-6c6b42200d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:47.919533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab11e06-8741-4f5d-a8e1-6c6b42200d7d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:47.921177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 570/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:42:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:47.927592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f45b3-5631-4af5-b331-d38ec6b587b0 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:48.024094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f45b3-5631-4af5-b331-d38ec6b587b0 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:48.057459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30462d4-2da4-4dcf-9ff3-4f31c564844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:48.058109 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30462d4-2da4-4dcf-9ff3-4f31c564844b None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:48.058392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30462d4-2da4-4dcf-9ff3-4f31c564844b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:48.058668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30462d4-2da4-4dcf-9ff3-4f31c564844b None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:48.059364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30462d4-2da4-4dcf-9ff3-4f31c564844b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:48.060809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30462d4-2da4-4dcf-9ff3-4f31c564844b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:48.130549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 568/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:42:47 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:42:50.181023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e24e3-5bb3-4c1c-8b7c-2a49e9dd11f4 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:50.181552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e24e3-5bb3-4c1c-8b7c-2a49e9dd11f4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:50.181646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e24e3-5bb3-4c1c-8b7c-2a49e9dd11f4 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:50.182945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 571/1139] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:50.188059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f788b-dd4a-42ba-b0d2-e2f9099befa6 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:50.188533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f788b-dd4a-42ba-b0d2-e2f9099befa6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:50.188643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f788b-dd4a-42ba-b0d2-e2f9099befa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:50.206363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-607f788b-dd4a-42ba-b0d2-e2f9099befa6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:50.207526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-607f788b-dd4a-42ba-b0d2-e2f9099befa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:50.229797 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:50.271306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 569/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:42:50.294773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e24e3-5bb3-4c1c-8b7c-2a49e9dd11f4 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:50.309851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:50.392731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e24e3-5bb3-4c1c-8b7c-2a49e9dd11f4 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:50.425898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29079f98-2b1e-42af-ae37-7be2a0282a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:50.426667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29079f98-2b1e-42af-ae37-7be2a0282a15 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:50.426947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29079f98-2b1e-42af-ae37-7be2a0282a15 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:50.427220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29079f98-2b1e-42af-ae37-7be2a0282a15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:50.436899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29079f98-2b1e-42af-ae37-7be2a0282a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-29079f98-2b1e-42af-ae37-7be2a0282a15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:50.444686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:50.445829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:50.446784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 572/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:50.451707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f788b-dd4a-42ba-b0d2-e2f9099befa6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:50.547867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f788b-dd4a-42ba-b0d2-e2f9099befa6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:50.581077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:50.581735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:50.582018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:50.582294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:50.583029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:50.583335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:50.584846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:50.592524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 570/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:50.598885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29079f98-2b1e-42af-ae37-7be2a0282a15 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:50.667271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:50.701253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29079f98-2b1e-42af-ae37-7be2a0282a15 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:50.729877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9db51-0eed-44d4-827d-d2805ee2282d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:50.730735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9db51-0eed-44d4-827d-d2805ee2282d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:50.731021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9db51-0eed-44d4-827d-d2805ee2282d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:50.731298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9db51-0eed-44d4-827d-d2805ee2282d None admin] PATH_INFO: `/v3/users/ffd3294ae94f42ae9cc3be3abb38d0bf` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:50.744050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9db51-0eed-44d4-827d-d2805ee2282d None admin] RBAC: Authorizing `identity:get_user(user_id=ffd3294ae94f42ae9cc3be3abb38d0bf)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:50.745510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb9db51-0eed-44d4-827d-d2805ee2282d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:50.747136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 573/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:50 2022] GET /identity/v3/users/ffd3294ae94f42ae9cc3be3abb38d0bf => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:50.752924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:50.775403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:50.851189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620efda2-427f-4cae-b5ec-02408d66ab5b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:50.883913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac52ba9-471c-4530-b360-79192497bc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:50.884586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac52ba9-471c-4530-b360-79192497bc1c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:50.884873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac52ba9-471c-4530-b360-79192497bc1c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:50.885145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac52ba9-471c-4530-b360-79192497bc1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:50.950829 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac52ba9-471c-4530-b360-79192497bc1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:50.952343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac52ba9-471c-4530-b360-79192497bc1c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:50.972557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:51.046626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 571/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:51.053312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9db51-0eed-44d4-827d-d2805ee2282d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:51.149843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9db51-0eed-44d4-827d-d2805ee2282d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:51.178727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:51.184353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44eea4e-4f66-43db-a37c-9fcdfe3e6544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:51.185204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eea4e-4f66-43db-a37c-9fcdfe3e6544 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:51.185489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eea4e-4f66-43db-a37c-9fcdfe3e6544 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:51.185767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eea4e-4f66-43db-a37c-9fcdfe3e6544 None admin] PATH_INFO: `/v3/projects/a993bf5367ad4667857bf8f21e06afd2` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:51.187112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44eea4e-4f66-43db-a37c-9fcdfe3e6544 None admin] RBAC: Authorizing `identity:get_project(project_id=a993bf5367ad4667857bf8f21e06afd2)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:51.188577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44eea4e-4f66-43db-a37c-9fcdfe3e6544 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:51.190315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 574/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:51 2022] GET /identity/v3/projects/a993bf5367ad4667857bf8f21e06afd2 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:51.196624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac52ba9-471c-4530-b360-79192497bc1c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:51.291366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac52ba9-471c-4530-b360-79192497bc1c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:51.310391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:51.323676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928805f1-59f7-42b5-a9ea-2511ecaa3d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:51.324349 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928805f1-59f7-42b5-a9ea-2511ecaa3d9d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:51.325169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928805f1-59f7-42b5-a9ea-2511ecaa3d9d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:51.325169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928805f1-59f7-42b5-a9ea-2511ecaa3d9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:51.325786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928805f1-59f7-42b5-a9ea-2511ecaa3d9d None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=a993bf5367ad4667857bf8f21e06afd2, user.id=ffd3294ae94f42ae9cc3be3abb38d0bf) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:51.326096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928805f1-59f7-42b5-a9ea-2511ecaa3d9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:51.327525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928805f1-59f7-42b5-a9ea-2511ecaa3d9d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:51.332961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 572/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:51 2022] GET /identity/v3/role_assignments?user.id=ffd3294ae94f42ae9cc3be3abb38d0bf&scope.project.id=a993bf5367ad4667857bf8f21e06afd2&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:53.326011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a529df4-d964-47b9-a17a-c7a25f48b82f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:53.326559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a529df4-d964-47b9-a17a-c7a25f48b82f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:53.326559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a529df4-d964-47b9-a17a-c7a25f48b82f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:53.327939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 575/1147] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:53.333599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49140b4-0a01-4df7-aa81-06cab1ebd2ac None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:53.333989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49140b4-0a01-4df7-aa81-06cab1ebd2ac None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:53.334148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49140b4-0a01-4df7-aa81-06cab1ebd2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:53.351109 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a49140b4-0a01-4df7-aa81-06cab1ebd2ac None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:53.352311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a49140b4-0a01-4df7-aa81-06cab1ebd2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:53.415794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 573/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:53.437785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a529df4-d964-47b9-a17a-c7a25f48b82f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:53.536216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a529df4-d964-47b9-a17a-c7a25f48b82f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:53.568980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac673-b3ca-4655-8db5-20fba54ce36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:53.569755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac673-b3ca-4655-8db5-20fba54ce36e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:53.570036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac673-b3ca-4655-8db5-20fba54ce36e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:53.570309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0ac673-b3ca-4655-8db5-20fba54ce36e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:53.580066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0ac673-b3ca-4655-8db5-20fba54ce36e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-1d0ac673-b3ca-4655-8db5-20fba54ce36e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:53.588250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:53.589336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:42:53.590365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 576/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:53.595419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49140b4-0a01-4df7-aa81-06cab1ebd2ac None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:53.689225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49140b4-0a01-4df7-aa81-06cab1ebd2ac None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:53.703389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:53.723269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:53.723938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:53.726731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:53.726731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:53.726731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:53.726731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:53.727127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:53.734821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 574/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:53.741226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac673-b3ca-4655-8db5-20fba54ce36e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:53.799501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:53.840069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0ac673-b3ca-4655-8db5-20fba54ce36e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:53.872691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6714b-e0b7-4554-9757-17eff3718e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:53.873591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6714b-e0b7-4554-9757-17eff3718e96 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:53.873876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6714b-e0b7-4554-9757-17eff3718e96 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:53.874153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6714b-e0b7-4554-9757-17eff3718e96 None admin] PATH_INFO: `/v3/users/ffd3294ae94f42ae9cc3be3abb38d0bf` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:53.875341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6714b-e0b7-4554-9757-17eff3718e96 None admin] RBAC: Authorizing `identity:get_user(user_id=ffd3294ae94f42ae9cc3be3abb38d0bf)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:53.876820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6714b-e0b7-4554-9757-17eff3718e96 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:53.879198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 577/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:53 2022] GET /identity/v3/users/ffd3294ae94f42ae9cc3be3abb38d0bf => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:53.885368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:53.982558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48850754-c0ad-4af3-aeef-35537aa1389a None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:54.014638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e3ac1-0f58-45bc-ba3d-823df0fafd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:54.015487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e3ac1-0f58-45bc-ba3d-823df0fafd72 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:54.015770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e3ac1-0f58-45bc-ba3d-823df0fafd72 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:54.016049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e3ac1-0f58-45bc-ba3d-823df0fafd72 None admin] PATH_INFO: `/v3/projects/a993bf5367ad4667857bf8f21e06afd2` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:54.017254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e3ac1-0f58-45bc-ba3d-823df0fafd72 None admin] RBAC: Authorizing `identity:get_project(project_id=a993bf5367ad4667857bf8f21e06afd2)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:54.018707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e3ac1-0f58-45bc-ba3d-823df0fafd72 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:54.020418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 575/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:53 2022] GET /identity/v3/projects/a993bf5367ad4667857bf8f21e06afd2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:54.026303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6714b-e0b7-4554-9757-17eff3718e96 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:54.124660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6714b-e0b7-4554-9757-17eff3718e96 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:54.149143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:54.157559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b374f6f6-2702-446c-823a-4d9888723ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:54.158955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f6f6-2702-446c-823a-4d9888723ca2 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:54.161348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f6f6-2702-446c-823a-4d9888723ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:54.161348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b374f6f6-2702-446c-823a-4d9888723ca2 None admin] PATH_INFO: `/v3/projects/a993bf5367ad4667857bf8f21e06afd2/users/ffd3294ae94f42ae9cc3be3abb38d0bf/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:54.161527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374f6f6-2702-446c-823a-4d9888723ca2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a993bf5367ad4667857bf8f21e06afd2, user_id=ffd3294ae94f42ae9cc3be3abb38d0bf, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:54.162988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b374f6f6-2702-446c-823a-4d9888723ca2 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:54.228192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 578/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:42:54 2022] PUT /identity/v3/projects/a993bf5367ad4667857bf8f21e06afd2/users/ffd3294ae94f42ae9cc3be3abb38d0bf/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:42:56.259989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a79c0e-e356-4b20-b6c2-878d0ee93ad4 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:56.260641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a79c0e-e356-4b20-b6c2-878d0ee93ad4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:56.260641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a79c0e-e356-4b20-b6c2-878d0ee93ad4 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:56.261958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 576/1154] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:56.267203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345902c7-f83e-4580-a5dd-9984e5081e14 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:56.267710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345902c7-f83e-4580-a5dd-9984e5081e14 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:56.267833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345902c7-f83e-4580-a5dd-9984e5081e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:56.284851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-345902c7-f83e-4580-a5dd-9984e5081e14 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:56.286041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-345902c7-f83e-4580-a5dd-9984e5081e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:56.351158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 579/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:56.374582 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a79c0e-e356-4b20-b6c2-878d0ee93ad4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:56.419858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:56.473462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a79c0e-e356-4b20-b6c2-878d0ee93ad4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:56.505431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6f38e-7561-46ad-85be-1f1c6c3a6c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:56.506207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6f38e-7561-46ad-85be-1f1c6c3a6c40 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:56.506491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6f38e-7561-46ad-85be-1f1c6c3a6c40 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:56.506768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6f38e-7561-46ad-85be-1f1c6c3a6c40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:56.516593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6f38e-7561-46ad-85be-1f1c6c3a6c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-eba6f38e-7561-46ad-85be-1f1c6c3a6c40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:42:56.524529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:42:56.525633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:42:56.526696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 577/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:42:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:42:56.531574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345902c7-f83e-4580-a5dd-9984e5081e14 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:56.628426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345902c7-f83e-4580-a5dd-9984e5081e14 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:56.644628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:56.662146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:56.662810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:56.663093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:56.663367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:56.664131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:56.664455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:56.665823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:56.673706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 580/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:42:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:56.679540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6f38e-7561-46ad-85be-1f1c6c3a6c40 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:56.776794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6f38e-7561-46ad-85be-1f1c6c3a6c40 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:56.779062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:56.810583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9f394-97a4-4180-82bb-678196272687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:56.811486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9f394-97a4-4180-82bb-678196272687 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:56.812048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9f394-97a4-4180-82bb-678196272687 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:56.812353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9f394-97a4-4180-82bb-678196272687 None admin] PATH_INFO: `/v3/users/ffd3294ae94f42ae9cc3be3abb38d0bf` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:56.813536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9f394-97a4-4180-82bb-678196272687 None admin] RBAC: Authorizing `identity:get_user(user_id=ffd3294ae94f42ae9cc3be3abb38d0bf)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:56.815690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9f394-97a4-4180-82bb-678196272687 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:56.817605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 578/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:42:56 2022] GET /identity/v3/users/ffd3294ae94f42ae9cc3be3abb38d0bf => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:42:56.823411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:56.919434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc0a7f-b9f6-4060-b7e6-5075a1b79f86 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:56.952599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f45f-4efe-4b8b-b7f3-d7b257ab1c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:56.953238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f45f-4efe-4b8b-b7f3-d7b257ab1c8c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:56.953520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f45f-4efe-4b8b-b7f3-d7b257ab1c8c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:56.953796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4f45f-4efe-4b8b-b7f3-d7b257ab1c8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:56.998525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:57.018946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f45f-4efe-4b8b-b7f3-d7b257ab1c8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:57.020346 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4f45f-4efe-4b8b-b7f3-d7b257ab1c8c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:57.115518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 581/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:57.121532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9f394-97a4-4180-82bb-678196272687 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:57.217030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9f394-97a4-4180-82bb-678196272687 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:57.251266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff5e80-478e-47ea-a5b5-738db51b587a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:57.252384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5e80-478e-47ea-a5b5-738db51b587a None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:57.252674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5e80-478e-47ea-a5b5-738db51b587a None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:57.252955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff5e80-478e-47ea-a5b5-738db51b587a None admin] PATH_INFO: `/v3/projects/a993bf5367ad4667857bf8f21e06afd2` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:57.254163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff5e80-478e-47ea-a5b5-738db51b587a None admin] RBAC: Authorizing `identity:get_project(project_id=a993bf5367ad4667857bf8f21e06afd2)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:57.255603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff5e80-478e-47ea-a5b5-738db51b587a None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:57.257334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 579/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:42:57 2022] GET /identity/v3/projects/a993bf5367ad4667857bf8f21e06afd2 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:57.263453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f45f-4efe-4b8b-b7f3-d7b257ab1c8c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:57.339446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:57.360189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4f45f-4efe-4b8b-b7f3-d7b257ab1c8c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:57.394130 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7560e42-3f61-40b0-8fb5-0aabe5c2e43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:57.394781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7560e42-3f61-40b0-8fb5-0aabe5c2e43d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:57.395073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7560e42-3f61-40b0-8fb5-0aabe5c2e43d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:57.395352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7560e42-3f61-40b0-8fb5-0aabe5c2e43d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:57.396221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7560e42-3f61-40b0-8fb5-0aabe5c2e43d None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=a993bf5367ad4667857bf8f21e06afd2, user.id=ffd3294ae94f42ae9cc3be3abb38d0bf) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:42:57.396539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7560e42-3f61-40b0-8fb5-0aabe5c2e43d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:57.397976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7560e42-3f61-40b0-8fb5-0aabe5c2e43d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:57.403658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 582/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:42:57 2022] GET /identity/v3/role_assignments?user.id=ffd3294ae94f42ae9cc3be3abb38d0bf&scope.project.id=a993bf5367ad4667857bf8f21e06afd2&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:42:59.430867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3f536-c6cd-4daf-81ba-f5f972784d57 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:59.431425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3f536-c6cd-4daf-81ba-f5f972784d57 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:59.431425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb3f536-c6cd-4daf-81ba-f5f972784d57 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:59.432830 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 580/1162] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:42:59.437994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed48ec7-4c65-4249-8ea3-1e7d41dbe1fb None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:59.438407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed48ec7-4c65-4249-8ea3-1e7d41dbe1fb None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:59.438544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed48ec7-4c65-4249-8ea3-1e7d41dbe1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:59.455532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-6ed48ec7-4c65-4249-8ea3-1e7d41dbe1fb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:42:59.456758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6ed48ec7-4c65-4249-8ea3-1e7d41dbe1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:42:59.522859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 583/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:42:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:42:59.546026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3f536-c6cd-4daf-81ba-f5f972784d57 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:59.587606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:59.643931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb3f536-c6cd-4daf-81ba-f5f972784d57 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:59.676251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4488e-e70d-4051-a528-2ea8e3307b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:59.676885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4488e-e70d-4051-a528-2ea8e3307b12 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:59.677166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4488e-e70d-4051-a528-2ea8e3307b12 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:59.677441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4488e-e70d-4051-a528-2ea8e3307b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:59.742680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4488e-e70d-4051-a528-2ea8e3307b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:59.744055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4488e-e70d-4051-a528-2ea8e3307b12 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:42:59.783483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:59.839277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 581/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:42:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:42:59.845295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed48ec7-4c65-4249-8ea3-1e7d41dbe1fb None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:42:59.913559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:42:59.941398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed48ec7-4c65-4249-8ea3-1e7d41dbe1fb None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:42:59.977110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212e929-7f1a-4b63-a099-b4325e129815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:42:59.977922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e929-7f1a-4b63-a099-b4325e129815 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:42:59.978209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e929-7f1a-4b63-a099-b4325e129815 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:42:59.978484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212e929-7f1a-4b63-a099-b4325e129815 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:42:59.985032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212e929-7f1a-4b63-a099-b4325e129815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:42:59.986762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212e929-7f1a-4b63-a099-b4325e129815 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-1212e929-7f1a-4b63-a099-b4325e129815 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:00.000731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:00.002520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:43:00.003654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 584/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:42:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:00.008436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4488e-e70d-4051-a528-2ea8e3307b12 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:00.104249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4488e-e70d-4051-a528-2ea8e3307b12 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:00.136765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629efb8-94b1-4cbd-8bd8-60d27e0ebf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:00.137415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629efb8-94b1-4cbd-8bd8-60d27e0ebf5e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:00.137699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629efb8-94b1-4cbd-8bd8-60d27e0ebf5e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:00.137983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629efb8-94b1-4cbd-8bd8-60d27e0ebf5e None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:00.138695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629efb8-94b1-4cbd-8bd8-60d27e0ebf5e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:00.139001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629efb8-94b1-4cbd-8bd8-60d27e0ebf5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:00.140387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629efb8-94b1-4cbd-8bd8-60d27e0ebf5e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:00.147920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 582/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Mon Jul 25 05:43:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:02.149256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c5fa-3197-42d3-a4f0-e78f537767ef None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:02.149779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c5fa-3197-42d3-a4f0-e78f537767ef None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:02.149779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607c5fa-3197-42d3-a4f0-e78f537767ef None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:02.151103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 585/1167] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:02.156559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0dff-b5c7-49a8-a25c-8ee3fe5cc88a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:02.156954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0dff-b5c7-49a8-a25c-8ee3fe5cc88a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:02.157122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad0dff-b5c7-49a8-a25c-8ee3fe5cc88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:02.163861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:02.174282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1aad0dff-b5c7-49a8-a25c-8ee3fe5cc88a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:02.175483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1aad0dff-b5c7-49a8-a25c-8ee3fe5cc88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:02.240221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 583/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:02.263216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607c5fa-3197-42d3-a4f0-e78f537767ef None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:02.361694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607c5fa-3197-42d3-a4f0-e78f537767ef None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:02.389646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:02.395039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8bebfd-469a-4d28-a47f-1084eb282c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:02.395687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bebfd-469a-4d28-a47f-1084eb282c98 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:02.395967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bebfd-469a-4d28-a47f-1084eb282c98 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:02.396270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8bebfd-469a-4d28-a47f-1084eb282c98 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:02.397022 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bebfd-469a-4d28-a47f-1084eb282c98 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:02.398379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8bebfd-469a-4d28-a47f-1084eb282c98 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:02.479595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 586/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Mon Jul 25 05:43:02 2022] POST /identity/v3/domains => generated 267 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:43:04.572887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6de3df-79b2-46eb-99a5-6927e491fa1f None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:04.573449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6de3df-79b2-46eb-99a5-6927e491fa1f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:04.573449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6de3df-79b2-46eb-99a5-6927e491fa1f None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:04.574787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 584/1170] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:04.579880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61439f8-0bbc-46cb-80ce-e96172fdadf0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:04.580339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61439f8-0bbc-46cb-80ce-e96172fdadf0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:04.580461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61439f8-0bbc-46cb-80ce-e96172fdadf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:04.597319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-a61439f8-0bbc-46cb-80ce-e96172fdadf0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:04.598507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a61439f8-0bbc-46cb-80ce-e96172fdadf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:04.662630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 587/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:04.686191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6de3df-79b2-46eb-99a5-6927e491fa1f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:04.785566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6de3df-79b2-46eb-99a5-6927e491fa1f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:04.818524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7446a-645f-444c-9542-e9f4d8c569a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:04.819539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7446a-645f-444c-9542-e9f4d8c569a4 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:04.819832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7446a-645f-444c-9542-e9f4d8c569a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:04.820104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7446a-645f-444c-9542-e9f4d8c569a4 None admin] PATH_INFO: `/v3/domains/e1cd7e5c8776485996a09c065619aadb` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:04.822183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:04.830839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7446a-645f-444c-9542-e9f4d8c569a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e1cd7e5c8776485996a09c065619aadb)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:04.832235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7446a-645f-444c-9542-e9f4d8c569a4 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:04.833849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 585/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Mon Jul 25 05:43:04 2022] GET /identity/v3/domains/e1cd7e5c8776485996a09c065619aadb => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:04.839642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61439f8-0bbc-46cb-80ce-e96172fdadf0 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:04.936573 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61439f8-0bbc-46cb-80ce-e96172fdadf0 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:04.959084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:04.970773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4887497-6a75-4088-a80a-f7678ba272fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:04.971719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4887497-6a75-4088-a80a-f7678ba272fd None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:04.971999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4887497-6a75-4088-a80a-f7678ba272fd None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:04.972302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4887497-6a75-4088-a80a-f7678ba272fd None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:04.973019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4887497-6a75-4088-a80a-f7678ba272fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:04.974450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4887497-6a75-4088-a80a-f7678ba272fd None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:05.062307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 588/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:43:04 2022] POST /identity/v3/projects => generated 361 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:43:07.104853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b217329-3ae1-4434-a771-427dcd35cc9d None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:07.105361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b217329-3ae1-4434-a771-427dcd35cc9d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:07.105361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b217329-3ae1-4434-a771-427dcd35cc9d None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:07.106687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 586/1174] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:07.112421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6358-a438-4695-8402-39657c209619 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:07.112795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6358-a438-4695-8402-39657c209619 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:07.112986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c6358-a438-4695-8402-39657c209619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:07.129370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-a37c6358-a438-4695-8402-39657c209619 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:07.130557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a37c6358-a438-4695-8402-39657c209619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:07.194874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 589/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:07.219390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b217329-3ae1-4434-a771-427dcd35cc9d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:07.319696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b217329-3ae1-4434-a771-427dcd35cc9d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:07.352615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8c87f-1aac-43db-a3b5-716452bf5afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:07.353473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8c87f-1aac-43db-a3b5-716452bf5afe None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:07.353756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8c87f-1aac-43db-a3b5-716452bf5afe None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:07.354034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8c87f-1aac-43db-a3b5-716452bf5afe None admin] PATH_INFO: `/v3/domains/e1cd7e5c8776485996a09c065619aadb` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:07.355258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8c87f-1aac-43db-a3b5-716452bf5afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=e1cd7e5c8776485996a09c065619aadb)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:07.356679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8c87f-1aac-43db-a3b5-716452bf5afe None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:07.358273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 587/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Mon Jul 25 05:43:07 2022] GET /identity/v3/domains/e1cd7e5c8776485996a09c065619aadb => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:07.364095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c6358-a438-4695-8402-39657c209619 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:07.459629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c6358-a438-4695-8402-39657c209619 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:07.461884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:07.492340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09910be7-d600-4def-a486-4ba725ee5273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:07.493000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09910be7-d600-4def-a486-4ba725ee5273 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:07.493284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09910be7-d600-4def-a486-4ba725ee5273 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:07.493559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09910be7-d600-4def-a486-4ba725ee5273 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:07.494270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09910be7-d600-4def-a486-4ba725ee5273 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:07.495809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09910be7-d600-4def-a486-4ba725ee5273 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:07.621266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 590/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:43:07 2022] POST /identity/v3/users => generated 311 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:43:09.702013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca99b-0a2a-43c0-83f9-302a63636302 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:09.702543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca99b-0a2a-43c0-83f9-302a63636302 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:09.702660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ca99b-0a2a-43c0-83f9-302a63636302 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:09.703932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 588/1178] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:09.709419 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f0c24-6d1d-4924-beec-e98a68e7e223 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:09.709829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f0c24-6d1d-4924-beec-e98a68e7e223 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:09.709983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55f0c24-6d1d-4924-beec-e98a68e7e223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:09.727640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d55f0c24-6d1d-4924-beec-e98a68e7e223 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:09.728872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d55f0c24-6d1d-4924-beec-e98a68e7e223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:09.794378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 591/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:09.818678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ca99b-0a2a-43c0-83f9-302a63636302 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:09.880529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:09.918719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ca99b-0a2a-43c0-83f9-302a63636302 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:09.951350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916aca-1580-4af9-8771-ea9352d26be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:09.952309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916aca-1580-4af9-8771-ea9352d26be7 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:09.952458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916aca-1580-4af9-8771-ea9352d26be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:09.952736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916aca-1580-4af9-8771-ea9352d26be7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:09.963589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0916aca-1580-4af9-8771-ea9352d26be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-d0916aca-1580-4af9-8771-ea9352d26be7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:09.972028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:09.973182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:43:09.974375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 589/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:43:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:09.979319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f0c24-6d1d-4924-beec-e98a68e7e223 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:10.066343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:10.076053 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55f0c24-6d1d-4924-beec-e98a68e7e223 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:10.109574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:10.110233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:10.111055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:10.111530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:10.112316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:10.112627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:10.113996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:10.121891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 592/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:43:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:10.127712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916aca-1580-4af9-8771-ea9352d26be7 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:10.224504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916aca-1580-4af9-8771-ea9352d26be7 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:10.247068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:10.258601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f842e-f9e9-4a0a-9fe3-c5025b2e398e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:10.259495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f842e-f9e9-4a0a-9fe3-c5025b2e398e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:10.259779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f842e-f9e9-4a0a-9fe3-c5025b2e398e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:10.260054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f842e-f9e9-4a0a-9fe3-c5025b2e398e None admin] PATH_INFO: `/v3/users/9f89f71c80b94d978201debd26f081c4` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:10.273387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f842e-f9e9-4a0a-9fe3-c5025b2e398e None admin] RBAC: Authorizing `identity:get_user(user_id=9f89f71c80b94d978201debd26f081c4)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:10.275973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943f842e-f9e9-4a0a-9fe3-c5025b2e398e None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:10.276600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 590/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:43:10 2022] GET /identity/v3/users/9f89f71c80b94d978201debd26f081c4 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:10.283146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:10.380133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337f5714-3999-4a9c-afa6-4a16d174a027 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:10.412920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9358b-5689-432c-a97c-5d653346b01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:10.413559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9358b-5689-432c-a97c-5d653346b01d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:10.413846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9358b-5689-432c-a97c-5d653346b01d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:10.414127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9358b-5689-432c-a97c-5d653346b01d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:10.434563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:10.480146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9358b-5689-432c-a97c-5d653346b01d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:10.481546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d9358b-5689-432c-a97c-5d653346b01d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:10.576394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 593/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:43:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:43:10.582910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f842e-f9e9-4a0a-9fe3-c5025b2e398e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:10.678057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f842e-f9e9-4a0a-9fe3-c5025b2e398e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:10.711072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a60f3-7832-4d7c-8687-68b12e549019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:10.711921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a60f3-7832-4d7c-8687-68b12e549019 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:10.712236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a60f3-7832-4d7c-8687-68b12e549019 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:10.712518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a60f3-7832-4d7c-8687-68b12e549019 None admin] PATH_INFO: `/v3/projects/6665bcfdbda542a5963a103c4e6f8bd9` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:10.713685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a60f3-7832-4d7c-8687-68b12e549019 None admin] RBAC: Authorizing `identity:get_project(project_id=6665bcfdbda542a5963a103c4e6f8bd9)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:10.715104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a60f3-7832-4d7c-8687-68b12e549019 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:10.716802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 591/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:43:10 2022] GET /identity/v3/projects/6665bcfdbda542a5963a103c4e6f8bd9 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:10.722922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9358b-5689-432c-a97c-5d653346b01d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:10.775279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:10.821518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9358b-5689-432c-a97c-5d653346b01d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:10.853617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f059254-3b4d-4e5c-98ec-2c7756001cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:10.854265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f059254-3b4d-4e5c-98ec-2c7756001cee None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:10.854548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f059254-3b4d-4e5c-98ec-2c7756001cee None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:10.854826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f059254-3b4d-4e5c-98ec-2c7756001cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:10.855659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f059254-3b4d-4e5c-98ec-2c7756001cee None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=6665bcfdbda542a5963a103c4e6f8bd9, user.id=9f89f71c80b94d978201debd26f081c4) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:10.855974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f059254-3b4d-4e5c-98ec-2c7756001cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:10.857433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f059254-3b4d-4e5c-98ec-2c7756001cee None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:10.863320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 594/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:43:10 2022] GET /identity/v3/role_assignments?user.id=9f89f71c80b94d978201debd26f081c4&scope.project.id=6665bcfdbda542a5963a103c4e6f8bd9&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:12.940256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8eef3-5448-4921-9e69-49657d2e3dd8 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:12.940805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8eef3-5448-4921-9e69-49657d2e3dd8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:12.940805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8eef3-5448-4921-9e69-49657d2e3dd8 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:12.942425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 592/1186] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:12.947627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab36657-182d-4055-af85-99415bf0e880 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:12.948045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab36657-182d-4055-af85-99415bf0e880 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:12.948203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab36657-182d-4055-af85-99415bf0e880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:12.964782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-bab36657-182d-4055-af85-99415bf0e880 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:12.965967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bab36657-182d-4055-af85-99415bf0e880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:13.031957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 595/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 25 05:43:13.055397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8eef3-5448-4921-9e69-49657d2e3dd8 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:13.105015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:13.154561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8eef3-5448-4921-9e69-49657d2e3dd8 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:13.187435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ab17-5d4d-421b-8ed4-97095fd4c1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:13.188283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ab17-5d4d-421b-8ed4-97095fd4c1fc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:13.188588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ab17-5d4d-421b-8ed4-97095fd4c1fc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:13.188995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ab17-5d4d-421b-8ed4-97095fd4c1fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:13.199462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4ab17-5d4d-421b-8ed4-97095fd4c1fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-1ff4ab17-5d4d-421b-8ed4-97095fd4c1fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:13.207681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:13.208898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:43:13.209834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 593/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:43:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:13.214959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab36657-182d-4055-af85-99415bf0e880 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:13.259830 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:13.311817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab36657-182d-4055-af85-99415bf0e880 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:13.345424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:13.346178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:13.346462 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:13.346739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:13.347489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:13.347798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:13.349206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:13.357040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 596/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:43:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:13.363445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ab17-5d4d-421b-8ed4-97095fd4c1fc None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:13.459806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ab17-5d4d-421b-8ed4-97095fd4c1fc None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:13.468218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:13.492196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996978-acc9-46f5-b87e-78e368752e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:13.493260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996978-acc9-46f5-b87e-78e368752e0f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:13.493382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996978-acc9-46f5-b87e-78e368752e0f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:13.493662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26996978-acc9-46f5-b87e-78e368752e0f None admin] PATH_INFO: `/v3/users/9f89f71c80b94d978201debd26f081c4` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:13.494838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26996978-acc9-46f5-b87e-78e368752e0f None admin] RBAC: Authorizing `identity:get_user(user_id=9f89f71c80b94d978201debd26f081c4)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:13.496294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26996978-acc9-46f5-b87e-78e368752e0f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:13.497905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 594/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:43:13 2022] GET /identity/v3/users/9f89f71c80b94d978201debd26f081c4 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:13.504268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:13.601512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f85e5e2-8544-4bda-affb-0cfcb17b2de3 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:13.609897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:13.634671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad808caa-36a6-4bb9-b0be-a8412d09d6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:13.635525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad808caa-36a6-4bb9-b0be-a8412d09d6ca None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:13.635806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad808caa-36a6-4bb9-b0be-a8412d09d6ca None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:13.636082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad808caa-36a6-4bb9-b0be-a8412d09d6ca None admin] PATH_INFO: `/v3/projects/6665bcfdbda542a5963a103c4e6f8bd9` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:13.637290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad808caa-36a6-4bb9-b0be-a8412d09d6ca None admin] RBAC: Authorizing `identity:get_project(project_id=6665bcfdbda542a5963a103c4e6f8bd9)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:13.638730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad808caa-36a6-4bb9-b0be-a8412d09d6ca None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:13.640498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 597/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:43:13 2022] GET /identity/v3/projects/6665bcfdbda542a5963a103c4e6f8bd9 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:13.647461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996978-acc9-46f5-b87e-78e368752e0f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:13.743951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26996978-acc9-46f5-b87e-78e368752e0f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:13.778158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f9513-be71-4153-af3d-ea9d41fffba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:13.779039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9513-be71-4153-af3d-ea9d41fffba4 None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:13.779319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9513-be71-4153-af3d-ea9d41fffba4 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:13.779593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f9513-be71-4153-af3d-ea9d41fffba4 None admin] PATH_INFO: `/v3/projects/6665bcfdbda542a5963a103c4e6f8bd9/users/9f89f71c80b94d978201debd26f081c4/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:13.781545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f9513-be71-4153-af3d-ea9d41fffba4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6665bcfdbda542a5963a103c4e6f8bd9, user_id=9f89f71c80b94d978201debd26f081c4, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:13.783265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f9513-be71-4153-af3d-ea9d41fffba4 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:13.838355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 595/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:43:13 2022] PUT /identity/v3/projects/6665bcfdbda542a5963a103c4e6f8bd9/users/9f89f71c80b94d978201debd26f081c4/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:43:15.843092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad924bfe-a6b0-49aa-b35f-4b3971920b47 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:15.843631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad924bfe-a6b0-49aa-b35f-4b3971920b47 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:15.843631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad924bfe-a6b0-49aa-b35f-4b3971920b47 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:15.845674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 598/1193] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:43:15.850977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a9a10-6963-45ba-951f-c37968ef67d5 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:15.851373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a9a10-6963-45ba-951f-c37968ef67d5 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:15.851538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a9a10-6963-45ba-951f-c37968ef67d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:15.868521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7c0a9a10-6963-45ba-951f-c37968ef67d5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:43:15.869704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7c0a9a10-6963-45ba-951f-c37968ef67d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:43:15.897102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:15.934327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 596/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:43:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:43:15.957582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad924bfe-a6b0-49aa-b35f-4b3971920b47 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:16.057067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad924bfe-a6b0-49aa-b35f-4b3971920b47 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:16.090100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb90d4-28c5-44eb-9953-c1fbc2adcea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:16.090893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb90d4-28c5-44eb-9953-c1fbc2adcea1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:16.091176 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb90d4-28c5-44eb-9953-c1fbc2adcea1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:16.091451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb90d4-28c5-44eb-9953-c1fbc2adcea1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:16.101857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb90d4-28c5-44eb-9953-c1fbc2adcea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-dbeb90d4-28c5-44eb-9953-c1fbc2adcea1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:43:16.109467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:43:16.111298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:43:16.112383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 599/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:43:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:43:16.119253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a9a10-6963-45ba-951f-c37968ef67d5 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:16.215411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a9a10-6963-45ba-951f-c37968ef67d5 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:16.248291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:16.248942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:16.249222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:16.249493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:16.250226 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:16.250590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:16.252377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:16.260003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 597/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:43:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:43:16.265596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb90d4-28c5-44eb-9953-c1fbc2adcea1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:16.301356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:16.364649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb90d4-28c5-44eb-9953-c1fbc2adcea1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:16.397235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4a91-9db6-4bc3-a812-f217e5c6b9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:16.398095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4a91-9db6-4bc3-a812-f217e5c6b9a8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:16.398375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4a91-9db6-4bc3-a812-f217e5c6b9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:16.398648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b4a91-9db6-4bc3-a812-f217e5c6b9a8 None admin] PATH_INFO: `/v3/users/9f89f71c80b94d978201debd26f081c4` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:16.399824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b4a91-9db6-4bc3-a812-f217e5c6b9a8 None admin] RBAC: Authorizing `identity:get_user(user_id=9f89f71c80b94d978201debd26f081c4)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:16.401323 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b4a91-9db6-4bc3-a812-f217e5c6b9a8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:16.402941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 600/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Mon Jul 25 05:43:16 2022] GET /identity/v3/users/9f89f71c80b94d978201debd26f081c4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:16.409331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:16.436992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:16.507869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde457b-d006-4c42-8d46-2bfceca45a48 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:16.541070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ecc8d-a619-438a-a0dd-5c3a14069845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:16.541698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecc8d-a619-438a-a0dd-5c3a14069845 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:16.541977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecc8d-a619-438a-a0dd-5c3a14069845 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:16.542252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ecc8d-a619-438a-a0dd-5c3a14069845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:16.607999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ecc8d-a619-438a-a0dd-5c3a14069845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:16.609452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ecc8d-a619-438a-a0dd-5c3a14069845 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:16.635691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:16.706601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 598/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:43:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:43:16.712499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4a91-9db6-4bc3-a812-f217e5c6b9a8 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:16.808208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b4a91-9db6-4bc3-a812-f217e5c6b9a8 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:16.810451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:16.842899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23a92a-4d8b-49bb-b125-938bcfc68858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:16.843820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23a92a-4d8b-49bb-b125-938bcfc68858 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:16.844441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23a92a-4d8b-49bb-b125-938bcfc68858 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:16.844944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23a92a-4d8b-49bb-b125-938bcfc68858 None admin] PATH_INFO: `/v3/projects/6665bcfdbda542a5963a103c4e6f8bd9` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:16.846119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23a92a-4d8b-49bb-b125-938bcfc68858 None admin] RBAC: Authorizing `identity:get_project(project_id=6665bcfdbda542a5963a103c4e6f8bd9)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:16.847541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23a92a-4d8b-49bb-b125-938bcfc68858 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:16.849240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 601/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Mon Jul 25 05:43:16 2022] GET /identity/v3/projects/6665bcfdbda542a5963a103c4e6f8bd9 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:16.855351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ecc8d-a619-438a-a0dd-5c3a14069845 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:43:16.954662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ecc8d-a619-438a-a0dd-5c3a14069845 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:43:16.979655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:16.988028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df7aeb-1233-4a78-91ba-995200f0735b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:43:16.994289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df7aeb-1233-4a78-91ba-995200f0735b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:43:16.994289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df7aeb-1233-4a78-91ba-995200f0735b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:43:16.994289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df7aeb-1233-4a78-91ba-995200f0735b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:43:16.994289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df7aeb-1233-4a78-91ba-995200f0735b None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=6665bcfdbda542a5963a103c4e6f8bd9, user.id=9f89f71c80b94d978201debd26f081c4) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:43:16.994289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df7aeb-1233-4a78-91ba-995200f0735b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:43:16.994289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df7aeb-1233-4a78-91ba-995200f0735b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:43:16.997576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 599/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:43:16 2022] GET /identity/v3/role_assignments?user.id=9f89f71c80b94d978201debd26f081c4&scope.project.id=6665bcfdbda542a5963a103c4e6f8bd9&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:43:26.866995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=14 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:43:27.009744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:27.756893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c265-f936-4b8a-b99c-0d4dd4c55a10 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:27.757470 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c265-f936-4b8a-b99c-0d4dd4c55a10 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:27.757470 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c265-f936-4b8a-b99c-0d4dd4c55a10 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:27.758799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 602/1201] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:27.764929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51662913-10e6-499b-a00f-452dc7dddc88 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:27.765376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51662913-10e6-499b-a00f-452dc7dddc88 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:27.765607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51662913-10e6-499b-a00f-452dc7dddc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:27.782242 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-51662913-10e6-499b-a00f-452dc7dddc88 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:27.783412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-51662913-10e6-499b-a00f-452dc7dddc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:27.849415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 600/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:27.874950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4c265-f936-4b8a-b99c-0d4dd4c55a10 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:27.978537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4c265-f936-4b8a-b99c-0d4dd4c55a10 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:28.011776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:28.012524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:28.012807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:28.013082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:28.023361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:28.024820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:28.041963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:28.043110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:28.050819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:28.050819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:28.050819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:28.050819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:28.050819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 603/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:28.051201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51662913-10e6-499b-a00f-452dc7dddc88 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:28.148049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51662913-10e6-499b-a00f-452dc7dddc88 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:28.176383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:28.182053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b50b790-2ac0-4658-a50c-3a5e359af90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:28.182752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50b790-2ac0-4658-a50c-3a5e359af90b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:28.183031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50b790-2ac0-4658-a50c-3a5e359af90b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:28.183303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b50b790-2ac0-4658-a50c-3a5e359af90b None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:28.184017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50b790-2ac0-4658-a50c-3a5e359af90b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:28.184355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50b790-2ac0-4658-a50c-3a5e359af90b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:28.186485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b50b790-2ac0-4658-a50c-3a5e359af90b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:28.197754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 601/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:28.203470 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:28.256090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:28.302179 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b68b93-5572-4500-831f-c903c70bb82a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:28.335378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec292401-8fd6-40f9-a21b-e5e258b7c29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:28.336042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec292401-8fd6-40f9-a21b-e5e258b7c29a None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:28.336350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec292401-8fd6-40f9-a21b-e5e258b7c29a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:28.336902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec292401-8fd6-40f9-a21b-e5e258b7c29a None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:28.337733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec292401-8fd6-40f9-a21b-e5e258b7c29a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:28.339904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec292401-8fd6-40f9-a21b-e5e258b7c29a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:28.411877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 604/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Mon Jul 25 05:44:28 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:44:30.469274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56fb71-81f6-44b6-8052-bd490a9a48c9 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.469946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56fb71-81f6-44b6-8052-bd490a9a48c9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.469946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56fb71-81f6-44b6-8052-bd490a9a48c9 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.471234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 602/1206] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:30.476433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95c2db-cacd-4fef-8169-643d804035db None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.476836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95c2db-cacd-4fef-8169-643d804035db None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.476975 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95c2db-cacd-4fef-8169-643d804035db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.494014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fa95c2db-cacd-4fef-8169-643d804035db None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:30.495185 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fa95c2db-cacd-4fef-8169-643d804035db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:30.560730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 605/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:30.584259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56fb71-81f6-44b6-8052-bd490a9a48c9 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.683636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56fb71-81f6-44b6-8052-bd490a9a48c9 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.716952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5556f0-bf6f-4bf3-86c0-defa089fa2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.717733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5556f0-bf6f-4bf3-86c0-defa089fa2d9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.718012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5556f0-bf6f-4bf3-86c0-defa089fa2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.718282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5556f0-bf6f-4bf3-86c0-defa089fa2d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.732700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5556f0-bf6f-4bf3-86c0-defa089fa2d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-ea5556f0-bf6f-4bf3-86c0-defa089fa2d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:44:30.736357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:30.737506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:30.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 603/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:44:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:30.743639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95c2db-cacd-4fef-8169-643d804035db None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.783517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:30.841979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95c2db-cacd-4fef-8169-643d804035db None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:30.875670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:30.876361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:30.876647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:30.876921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:30.877667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] RBAC: Adding query filter params (name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:30.877972 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:30.879392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:30.887229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 606/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:44:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:30.893031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5556f0-bf6f-4bf3-86c0-defa089fa2d9 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:30.944068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:30.992712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5556f0-bf6f-4bf3-86c0-defa089fa2d9 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.024383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.025171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.025455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.025733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.037305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.038904 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:31.056899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:31.062168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:31.063431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:31.063431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:31.063431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:31.063431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:31.063431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 604/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:31.064321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.160919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:31.163871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce04f831-eeeb-4071-9184-9b272435af44 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.197461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.198138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.198401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.198676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.199439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:31.199758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.202045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.213702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 607/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:44:31.220490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.326338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81ef1a-e974-4051-871f-4468729607dc None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.328299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:31.361297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.362090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.362379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.362656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.369480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.371609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:44:31.385401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:31.386521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:31.387752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 605/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:31.393085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.492423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da04c1-29f6-40e6-89d9-72bcaa714be7 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.526452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.527125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.527409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.527714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.528583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:31.528896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.530370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.547709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 608/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 25 05:44:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:31.553679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.652403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf64f6e-c253-43e5-8a68-ea1568998962 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.686054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b34c3f-1451-466b-befd-17005f2ecf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.686699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b34c3f-1451-466b-befd-17005f2ecf10 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.686981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b34c3f-1451-466b-befd-17005f2ecf10 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.687254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b34c3f-1451-466b-befd-17005f2ecf10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:31.717183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:31.753463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b34c3f-1451-466b-befd-17005f2ecf10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:31.754948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b34c3f-1451-466b-befd-17005f2ecf10 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:31.850950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 606/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:44:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:44:31.857333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:31.876966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:31.955285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5d3d7-2c31-4490-ac87-88e6d976f97d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:31.988475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:31.989261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:31.989542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:31.989818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:32.000297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:32.001792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:32.018956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:32.020027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:32.021434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:32.021434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:32.021434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:32.021434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:32.021434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 609/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:32.025807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b34c3f-1451-466b-befd-17005f2ecf10 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:32.087515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:32.126545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b34c3f-1451-466b-befd-17005f2ecf10 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:32.158873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:32.159539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:32.159825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:32.160103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:32.160853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:32.161162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:32.162543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:32.175122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 607/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:32.180978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:32.252503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:32.283635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16103a89-e50a-45aa-a3f0-6ad212ead799 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:32.319914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:32.320837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:32.321136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:32.321639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:32.329175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:32.331951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:44:32.344850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:32.345941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:32.346959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 610/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:44:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:32.352157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:32.457634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557829-9874-4d3f-88b6-edf9bb898720 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:32.480714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:32.492769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970df10-7415-48c4-9db4-2131b12a24e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:32.493438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970df10-7415-48c4-9db4-2131b12a24e0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:32.493720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970df10-7415-48c4-9db4-2131b12a24e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:32.493995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970df10-7415-48c4-9db4-2131b12a24e0 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:32.494782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970df10-7415-48c4-9db4-2131b12a24e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:32.495092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970df10-7415-48c4-9db4-2131b12a24e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:32.496557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970df10-7415-48c4-9db4-2131b12a24e0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:32.517578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 608/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:44:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:44:32.517747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:32.616876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e370945-2a4a-4c19-8c82-241d6c8fbe0e None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:32.644961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:32.650477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b235fa-8e12-411a-81b5-1c425072c38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:32.651126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b235fa-8e12-411a-81b5-1c425072c38a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:32.651408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b235fa-8e12-411a-81b5-1c425072c38a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:32.651718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b235fa-8e12-411a-81b5-1c425072c38a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:32.652588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b235fa-8e12-411a-81b5-1c425072c38a None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=7c4988dc5513476786f11d08059c3d13) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:32.652896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b235fa-8e12-411a-81b5-1c425072c38a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:32.655472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b235fa-8e12-411a-81b5-1c425072c38a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:32.661154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 611/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:44:32 2022] GET /identity/v3/role_assignments?user.id=7c4988dc5513476786f11d08059c3d13&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:34.639514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7dfde-043b-42ff-b83e-ba3c8fbe3f28 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:34.640097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7dfde-043b-42ff-b83e-ba3c8fbe3f28 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:34.640097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7dfde-043b-42ff-b83e-ba3c8fbe3f28 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:34.641442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 609/1220] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:34.646468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6555ff6-b93e-4d8f-8eb2-576b524af402 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:34.646889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6555ff6-b93e-4d8f-8eb2-576b524af402 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:34.647029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6555ff6-b93e-4d8f-8eb2-576b524af402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:34.663843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b6555ff6-b93e-4d8f-8eb2-576b524af402 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:34.665061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b6555ff6-b93e-4d8f-8eb2-576b524af402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:34.734990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 612/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:34.757932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7dfde-043b-42ff-b83e-ba3c8fbe3f28 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:34.860568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7dfde-043b-42ff-b83e-ba3c8fbe3f28 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:34.894459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab72d-832f-4874-9e43-63c1eba24f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:34.895188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab72d-832f-4874-9e43-63c1eba24f8e None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:34.895474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab72d-832f-4874-9e43-63c1eba24f8e None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:34.895777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ab72d-832f-4874-9e43-63c1eba24f8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:34.905915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ab72d-832f-4874-9e43-63c1eba24f8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-927ab72d-832f-4874-9e43-63c1eba24f8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:44:34.913877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:34.921186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:34.922214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 610/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:44:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:34.924350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6555ff6-b93e-4d8f-8eb2-576b524af402 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.021214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6555ff6-b93e-4d8f-8eb2-576b524af402 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.043674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:35.054922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.055590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.055872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.056172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.056965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] RBAC: Adding query filter params (name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:35.057273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.058645 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.067436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 613/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:44:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:35.073473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab72d-832f-4874-9e43-63c1eba24f8e None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.117396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:35.173267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ab72d-832f-4874-9e43-63c1eba24f8e None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.207504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.208311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.208593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.208866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.220091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.221553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:35.239470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:35.240620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:35.241883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:35.241883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:35.241883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:35.241883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:35.241883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 611/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:35.246758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.345539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4ba41-d23a-45cd-9f59-399559ba6681 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.379184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.379848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.380153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.380436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.381159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:35.381470 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.382930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.394447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 614/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:35.400580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.494261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:35.500406 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf3bfb-39ab-4d20-9838-1643a10a5a12 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.532572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.533363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.533652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.533928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.540764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.542287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:44:35.557708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:35.563518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:35.564720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 612/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:35.566060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.612488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:35.664068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab734bf-431f-4517-a64d-28749d204c19 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.697819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.698483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.698765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.699043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.699874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:35.700699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.702186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.719465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 615/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:35.725217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.825936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab67f06-c760-41bd-9e21-c592379a39d5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.858884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:35.859670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:35.859955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:35.860265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:35.869922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:35.871358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:35.888928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:35.896447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:35.897541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:35.897541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:35.897541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:35.897541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:35.897541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 613/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:35.897883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:35.994152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db694a-2de3-4385-94fb-250d4a522a50 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:35.996417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:36.027639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.028265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.028579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.028852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.029570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:36.029876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.031237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.042757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 616/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:36.048514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.057477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:36.146533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc476d7b-909f-4ea3-9e87-ec1819225f5b None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.180562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.181346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.181625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.181897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.189534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.190963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:44:36.205600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:36.209153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:36.210239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 614/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:44:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:36.212764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.311944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64827-c660-4fc0-a173-f31c4c142925 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.344419 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2983cc-2e0d-41d7-a7db-2b5066299faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.345079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2983cc-2e0d-41d7-a7db-2b5066299faf None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.345360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2983cc-2e0d-41d7-a7db-2b5066299faf None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.345634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2983cc-2e0d-41d7-a7db-2b5066299faf None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.346414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2983cc-2e0d-41d7-a7db-2b5066299faf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:36.346723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2983cc-2e0d-41d7-a7db-2b5066299faf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.348247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2983cc-2e0d-41d7-a7db-2b5066299faf None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.360662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 617/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:44:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:36.366726 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:36.431325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:36.467272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07476f4-ecee-4c77-8272-131fdfdc994d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:36.500990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6100d-9d3c-4861-ac7b-11dae45dbd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:36.501877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6100d-9d3c-4861-ac7b-11dae45dbd1b None admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:36.502163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6100d-9d3c-4861-ac7b-11dae45dbd1b None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:36.502440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6100d-9d3c-4861-ac7b-11dae45dbd1b None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/7c4988dc5513476786f11d08059c3d13/roles/2c3d129fa9da4379acd24c88e17b7701` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:36.516347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6100d-9d3c-4861-ac7b-11dae45dbd1b None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=7c4988dc5513476786f11d08059c3d13, role_id=2c3d129fa9da4379acd24c88e17b7701)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:36.517804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6100d-9d3c-4861-ac7b-11dae45dbd1b None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:36.594285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 615/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:44:36 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/7c4988dc5513476786f11d08059c3d13/roles/2c3d129fa9da4379acd24c88e17b7701 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:44:38.645585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c59d9-614b-4e84-99df-2ee5749699e1 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:38.646115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c59d9-614b-4e84-99df-2ee5749699e1 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:38.646115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c59d9-614b-4e84-99df-2ee5749699e1 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:38.647521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 618/1233] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:38.652888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778c938-2503-4cc9-aa99-c8d8698ae8b7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:38.653837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778c938-2503-4cc9-aa99-c8d8698ae8b7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:38.654616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778c938-2503-4cc9-aa99-c8d8698ae8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:38.672408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-5778c938-2503-4cc9-aa99-c8d8698ae8b7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:38.673688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5778c938-2503-4cc9-aa99-c8d8698ae8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:38.740379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 616/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:38.765890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c59d9-614b-4e84-99df-2ee5749699e1 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:38.791924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:38.868148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c59d9-614b-4e84-99df-2ee5749699e1 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:38.902526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52924bc2-30ad-4a22-aa2e-25d3a5d38b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:38.903847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52924bc2-30ad-4a22-aa2e-25d3a5d38b06 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:38.904164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52924bc2-30ad-4a22-aa2e-25d3a5d38b06 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:38.904431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52924bc2-30ad-4a22-aa2e-25d3a5d38b06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:38.914351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52924bc2-30ad-4a22-aa2e-25d3a5d38b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-52924bc2-30ad-4a22-aa2e-25d3a5d38b06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:44:38.922126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:38.923676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:38.924907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 619/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Mon Jul 25 05:44:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:38.929436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778c938-2503-4cc9-aa99-c8d8698ae8b7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:38.974831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:39.027389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778c938-2503-4cc9-aa99-c8d8698ae8b7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:39.059946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:39.060615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:39.060898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:39.061178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:39.061924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] RBAC: Adding query filter params (name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:39.062232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:39.063854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:39.071483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 617/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Mon Jul 25 05:44:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:39.078303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52924bc2-30ad-4a22-aa2e-25d3a5d38b06 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:39.162257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:39.177702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52924bc2-30ad-4a22-aa2e-25d3a5d38b06 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:39.211120 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:39.211895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:39.212198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:39.212472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:39.222092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:39.223550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:39.240461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:39.241502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:39.242409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:39.242409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:39.242409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:39.242409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:39.242409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 620/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:39.247960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:39.346820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c78db-901c-4012-b1a9-bc277437a529 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:39.355446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:39.380377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:39.381039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:39.381321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:39.381598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:39.382316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:39.382626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:39.383985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:39.395270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 618/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:39.401016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:39.501983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcac176-c0f2-48e2-97df-48b94eb03fe7 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:39.535927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:39.536778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:39.537064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:39.542107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] PATH_INFO: `/v3/users/placement` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:39.542107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:39.545526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:39.547029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:44:39.562569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:39.567821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:39.569038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 621/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:39.570829 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:39.669735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e025a5a-b06a-45b5-9480-815d0ec1fb01 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:39.703720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:39.704423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:39.704706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:39.704979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:39.705801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:39.706128 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:39.707525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:39.724855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 619/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 25 05:44:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:39.730944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:39.831343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25885cd0-4891-48a1-b10f-048c771ce67d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:39.866540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bab38-8b4b-4ba3-bb8b-6a01aebea1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:39.867185 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bab38-8b4b-4ba3-bb8b-6a01aebea1fd None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:39.867466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bab38-8b4b-4ba3-bb8b-6a01aebea1fd None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:39.867786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bab38-8b4b-4ba3-bb8b-6a01aebea1fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:39.927456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:39.933792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bab38-8b4b-4ba3-bb8b-6a01aebea1fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:39.935309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bab38-8b4b-4ba3-bb8b-6a01aebea1fd None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:40.028968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 622/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:44:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:44:40.034853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:40.062507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:40.131026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8b76f-6f27-4015-aa35-85d5a47a2861 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:40.165462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:40.166504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:40.166990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:40.167854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:40.178326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:40.179876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:40.197346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:40.198410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:40.199379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:40.199379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:40.199379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:40.199379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:40.199379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 620/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:40.205062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bab38-8b4b-4ba3-bb8b-6a01aebea1fd None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:40.296370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:40.302314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bab38-8b4b-4ba3-bb8b-6a01aebea1fd None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:40.335371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:40.336029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:40.336337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:40.336619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:40.337340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:40.337646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:40.339004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:40.351249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 623/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:40.357522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:40.432862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:40.454568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe0481-34f4-4a97-ac42-1f501610b510 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:40.486754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:40.487548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:40.487833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:40.488122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:40.494655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:40.496395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:44:40.510275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:40.511341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:40.513211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 621/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:44:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:40.519519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:40.619246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3621db-fcda-4ba5-b7ba-bbdb0a2349b0 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:40.652990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27a616-fac8-4c55-b3e4-77c5782a7ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:40.653643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a616-fac8-4c55-b3e4-77c5782a7ae9 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:40.653926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a616-fac8-4c55-b3e4-77c5782a7ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:40.654196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a616-fac8-4c55-b3e4-77c5782a7ae9 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:40.654977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27a616-fac8-4c55-b3e4-77c5782a7ae9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:40.655283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27a616-fac8-4c55-b3e4-77c5782a7ae9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:40.656742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27a616-fac8-4c55-b3e4-77c5782a7ae9 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:40.668572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 624/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:44:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:40.674866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:40.778965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570186a7-48a8-411a-8e85-c32d77768500 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:40.815157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5653eb4-d341-4504-87ac-df5be0aabea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:40.815812 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5653eb4-d341-4504-87ac-df5be0aabea6 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:40.816098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5653eb4-d341-4504-87ac-df5be0aabea6 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:40.816401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5653eb4-d341-4504-87ac-df5be0aabea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:40.817229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5653eb4-d341-4504-87ac-df5be0aabea6 None admin] RBAC: Adding query filter params (role.id=2c3d129fa9da4379acd24c88e17b7701, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=7c4988dc5513476786f11d08059c3d13) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:40.817539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5653eb4-d341-4504-87ac-df5be0aabea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:40.818963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5653eb4-d341-4504-87ac-df5be0aabea6 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:40.824736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 622/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:44:40 2022] GET /identity/v3/role_assignments?user.id=7c4988dc5513476786f11d08059c3d13&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=2c3d129fa9da4379acd24c88e17b7701 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:42.943043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7780f-3781-49f9-b85e-7982378c51f6 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:42.943631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7780f-3781-49f9-b85e-7982378c51f6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:42.943631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc7780f-3781-49f9-b85e-7982378c51f6 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:42.944999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 625/1247] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:42.950135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6290b9-e9f2-4ca8-aad2-18bf00b0ec43 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:42.950543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6290b9-e9f2-4ca8-aad2-18bf00b0ec43 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:42.951331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6290b9-e9f2-4ca8-aad2-18bf00b0ec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:42.956461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:42.968285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-da6290b9-e9f2-4ca8-aad2-18bf00b0ec43 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:42.969549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-da6290b9-e9f2-4ca8-aad2-18bf00b0ec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:43.033904 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 623/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:43.057882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7780f-3781-49f9-b85e-7982378c51f6 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:43.076849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:43.158718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc7780f-3781-49f9-b85e-7982378c51f6 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:43.193723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1d75c-d722-4fb6-890e-01037061bf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:43.194513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1d75c-d722-4fb6-890e-01037061bf37 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:43.194798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1d75c-d722-4fb6-890e-01037061bf37 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:43.195074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1d75c-d722-4fb6-890e-01037061bf37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:43.205242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1d75c-d722-4fb6-890e-01037061bf37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-cda1d75c-d722-4fb6-890e-01037061bf37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:44:43.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:43.214375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:43.215290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 626/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:44:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:43.221108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6290b9-e9f2-4ca8-aad2-18bf00b0ec43 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:43.318518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6290b9-e9f2-4ca8-aad2-18bf00b0ec43 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:43.346969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:43.351728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:43.352421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:43.352703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:43.352976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:43.353714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:43.354019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:43.355385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:43.363853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 624/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:44:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:43.369632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1d75c-d722-4fb6-890e-01037061bf37 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:43.446734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:43.468572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1d75c-d722-4fb6-890e-01037061bf37 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:43.501488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:43.502296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:43.502594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:43.502854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:43.513705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:43.515189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:43.532608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:43.533710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:43.534919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:43.534919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:43.534919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:43.534919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:43.536272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 627/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:43.540482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:43.638708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8df64-eab8-476d-8ab1-413c8d62da98 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:43.671837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:43.672532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:43.672839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:43.673125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:43.673843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:43.674151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:43.675524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:43.688283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 625/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:43.695148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:43.793330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6e230-22ff-41eb-8a11-039ebc5354c6 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:43.818568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:43.827206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:43.828004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:43.828319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:43.828592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] PATH_INFO: `/v3/users/placement` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:43.836063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:43.837563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:44:43.852473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:43.853826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:43.854896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 628/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:43.859632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:43.915481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:43.958545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aada0995-d263-4a13-8034-e255b8d2a6d7 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:43.991224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:43.991886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:43.992259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:43.992540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:43.993416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:43.993726 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:43.995173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:44.011684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 626/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 25 05:44:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:44.018211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:44.116469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72130e-5bb3-420c-91d8-e03e9de8b93a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:44.149386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cb9fb-2549-49e0-be45-59acd76a18ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:44.150027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb9fb-2549-49e0-be45-59acd76a18ed None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:44.150310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb9fb-2549-49e0-be45-59acd76a18ed None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:44.150585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cb9fb-2549-49e0-be45-59acd76a18ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:44.206997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:44.218940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cb9fb-2549-49e0-be45-59acd76a18ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:44.220424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cb9fb-2549-49e0-be45-59acd76a18ed None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:44.317458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 629/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:44:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 25 05:44:44.323311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:44.421692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22148a2f-db69-4c83-8c93-9ef165c3d574 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:44.434148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:44.455787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:44.462033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:44.462033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:44.462033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:44.466987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:44.468519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:44.485755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:44.491418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:44.492417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:44.492417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:44.492417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:44.492417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:44.492417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 627/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:44.492949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cb9fb-2549-49e0-be45-59acd76a18ed None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:44.580521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:44.592560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cb9fb-2549-49e0-be45-59acd76a18ed None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:44.627284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:44.627920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:44.628227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:44.628503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:44.629221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:44.629523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:44.630894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:44.641880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 630/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:44:44.647894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:44.746708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8940ae-294a-47f8-8e6a-e97fc81a31ed None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:44.781196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:44.781997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:44.782276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:44.782550 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:44.789085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:44.790515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:44:44.805014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:44.806086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:44.807001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 628/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:44:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:44.813448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:44.916834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75e20e-7611-497d-a578-09b259eacd64 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:44.950088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107cc15-a13f-4ca4-beec-c7086850cc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:44.950769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107cc15-a13f-4ca4-beec-c7086850cc71 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:44.951051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107cc15-a13f-4ca4-beec-c7086850cc71 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:44.951325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107cc15-a13f-4ca4-beec-c7086850cc71 None admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:44.952101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107cc15-a13f-4ca4-beec-c7086850cc71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:44.952501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107cc15-a13f-4ca4-beec-c7086850cc71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:44.953944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107cc15-a13f-4ca4-beec-c7086850cc71 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:44.965694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 631/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:44:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:44.972569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:44.998672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:45.071013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d5f796-78d5-445b-a5b0-c18049f09133 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:45.104161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc3bdc-cad4-495d-9068-546d8887a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:45.104811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3bdc-cad4-495d-9068-546d8887a75d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:45.105097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3bdc-cad4-495d-9068-546d8887a75d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:45.105376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc3bdc-cad4-495d-9068-546d8887a75d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:45.106203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc3bdc-cad4-495d-9068-546d8887a75d None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=7c4988dc5513476786f11d08059c3d13) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:45.106519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc3bdc-cad4-495d-9068-546d8887a75d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:45.108695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc3bdc-cad4-495d-9068-546d8887a75d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:45.114048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 629/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:44:44 2022] GET /identity/v3/role_assignments?user.id=7c4988dc5513476786f11d08059c3d13&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:47.126973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63894b8-4405-4c40-b4c8-1004c20dfb2a None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:47.127471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63894b8-4405-4c40-b4c8-1004c20dfb2a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:47.127471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63894b8-4405-4c40-b4c8-1004c20dfb2a None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:47.128859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 632/1261] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:44:47.134066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449b5c-7a05-4042-b3c0-48bb7263a4e7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:47.134491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449b5c-7a05-4042-b3c0-48bb7263a4e7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:47.134627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449b5c-7a05-4042-b3c0-48bb7263a4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:47.152038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-77449b5c-7a05-4042-b3c0-48bb7263a4e7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:47.153314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-77449b5c-7a05-4042-b3c0-48bb7263a4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:47.218997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 630/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:47.242287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63894b8-4405-4c40-b4c8-1004c20dfb2a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:47.253315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:47.345072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63894b8-4405-4c40-b4c8-1004c20dfb2a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:47.378697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc012f-e71a-4647-8396-c07e78d77ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:47.379419 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc012f-e71a-4647-8396-c07e78d77ba1 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:47.379704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc012f-e71a-4647-8396-c07e78d77ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:47.379981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc012f-e71a-4647-8396-c07e78d77ba1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:47.390184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc012f-e71a-4647-8396-c07e78d77ba1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-43fc012f-e71a-4647-8396-c07e78d77ba1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:44:47.397751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:47.398782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:47.399803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 633/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:44:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:47.405940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77449b5c-7a05-4042-b3c0-48bb7263a4e7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:47.424910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:47.504443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77449b5c-7a05-4042-b3c0-48bb7263a4e7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:47.538197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:47.538859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:47.539141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:47.539419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:47.540181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:47.540488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:47.541845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:47.549896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 631/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:44:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:47.555832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc012f-e71a-4647-8396-c07e78d77ba1 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:47.621664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:47.661597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc012f-e71a-4647-8396-c07e78d77ba1 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:47.693962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:47.694745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:47.695023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:47.695295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:47.706756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:47.708231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:47.725220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:47.731578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:47.732892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:47.732892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:47.732892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:47.732892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:47.732892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 634/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:47.733243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:47.815164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:47.834166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efc85d-0834-46fe-9876-bd50a195165d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:47.866551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:47.867216 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:47.867501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:47.867781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:47.868539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:47.868850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:47.870222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:47.881044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 632/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:47.887059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:47.989183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b46aa4-0402-453b-b90f-d93d3f3f816a None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:48.009304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:48.023709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:48.024531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.024821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.025101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.032759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:48.034287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:44:48.048508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:48.049624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:48.050802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 635/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:48.055865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.156507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d940072-a357-4b97-b236-4a91d4f0d813 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:48.189694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:48.190357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.190642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.190919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.191742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:48.192048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:48.193487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:48.202866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:48.210547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 633/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 25 05:44:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:48.216800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.320786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd8a99-8eca-4ff3-8034-d9b6d77d6225 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:48.355865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:48.357199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.357557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.357834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.368158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:48.369617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:48.386667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:48.387722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:48.388760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:48.388760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:48.388760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:48.388760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:48.388760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 636/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:48.394096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.492916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142dd70-55ef-4104-a6b8-d1687c53211c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:48.527751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:48.528496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.528833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.529115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.529963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:48.530271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:48.531641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:48.542904 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 634/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:44:48.549788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.581944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:48.649547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e777ea4c-3335-4e42-b520-c117a2c077d9 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:48.682701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:48.683483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.683759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.684026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.691330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:48.692787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:44:48.706646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:48.707790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:48.708834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 637/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:44:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:48.713833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.774067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:48.815452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083ff8d-cfc8-4054-85a4-0a27bb506635 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:48.848171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089334f-135b-45a1-84bc-b6f2f06fd384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:48.848770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089334f-135b-45a1-84bc-b6f2f06fd384 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:48.849056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089334f-135b-45a1-84bc-b6f2f06fd384 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:48.849332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089334f-135b-45a1-84bc-b6f2f06fd384 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:48.850135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089334f-135b-45a1-84bc-b6f2f06fd384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:48.850536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089334f-135b-45a1-84bc-b6f2f06fd384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:48.852034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089334f-135b-45a1-84bc-b6f2f06fd384 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:48.864632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 635/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:44:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:44:48.870778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:48.961381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:48.969584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141fe3ab-efd7-4fef-b148-61fbccaef591 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:49.002124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199dca9-fc2f-4662-ab8f-cc5b39745920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:49.003002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199dca9-fc2f-4662-ab8f-cc5b39745920 None admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:49.003278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199dca9-fc2f-4662-ab8f-cc5b39745920 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:49.003552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199dca9-fc2f-4662-ab8f-cc5b39745920 None admin] PATH_INFO: `/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/7c4988dc5513476786f11d08059c3d13/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:49.005717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199dca9-fc2f-4662-ab8f-cc5b39745920 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca1bb14e085e4c8ab908e3880a657645, user_id=7c4988dc5513476786f11d08059c3d13, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:49.007135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2199dca9-fc2f-4662-ab8f-cc5b39745920 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:49.080580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 638/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Mon Jul 25 05:44:48 2022] PUT /identity/v3/projects/ca1bb14e085e4c8ab908e3880a657645/users/7c4988dc5513476786f11d08059c3d13/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:44:51.103213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4545-0435-436f-b443-2abf20e643cc None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.103762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4545-0435-436f-b443-2abf20e643cc None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.103762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb4545-0435-436f-b443-2abf20e643cc None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.105093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 636/1274] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:51.110433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b85d55-c9f1-4fb2-afe2-832efa352e5f None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.110889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b85d55-c9f1-4fb2-afe2-832efa352e5f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.110983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b85d55-c9f1-4fb2-afe2-832efa352e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.128364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-43b85d55-c9f1-4fb2-afe2-832efa352e5f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:51.129552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-43b85d55-c9f1-4fb2-afe2-832efa352e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:51.194885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 639/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:51.218934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb4545-0435-436f-b443-2abf20e643cc None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.318912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb4545-0435-436f-b443-2abf20e643cc None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.335371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:51.352084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e876-8787-4710-b26e-1c3d7d26144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.352966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e876-8787-4710-b26e-1c3d7d26144d None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.353301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e876-8787-4710-b26e-1c3d7d26144d None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.353528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d7e876-8787-4710-b26e-1c3d7d26144d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.363678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d7e876-8787-4710-b26e-1c3d7d26144d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-23d7e876-8787-4710-b26e-1c3d7d26144d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 25 05:44:51.371324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:51.373504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:51.374369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 637/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Mon Jul 25 05:44:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:51.379248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b85d55-c9f1-4fb2-afe2-832efa352e5f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.460929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:51.478617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b85d55-c9f1-4fb2-afe2-832efa352e5f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.513316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.513973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.514255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.514529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.515274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:51.515592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.516964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.525401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 640/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Mon Jul 25 05:44:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:51.531054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e876-8787-4710-b26e-1c3d7d26144d None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.630440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d7e876-8787-4710-b26e-1c3d7d26144d None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.663502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.664325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.664616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.664895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.676175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.677663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:51.694854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:51.699248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:51.700613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:51.700613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:51.700613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:51.700613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:51.700613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 638/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:51.702435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.802790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89afa1df-6588-446f-a4f0-8812e19dab89 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.835233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.835890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.836202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.836479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.837200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:51.837507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.838870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:51.850747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 641/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:51.857405 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:51.888232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:51.955480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517a9aa-7266-4005-a9e8-33d6dc63fa4c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:51.989527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:51.990324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:51.990605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:51.990878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] PATH_INFO: `/v3/users/placement` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:51.998014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:51.999457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 25 05:44:52.014564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:52.015665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:52.016876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 639/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:52.021659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:52.025409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:52.120755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e159f34-708b-4e8d-b4d3-6b4165a6158c None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:52.157925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:52.157925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:52.157925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:52.157925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:52.158343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:52.158343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:52.158515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:52.176651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 642/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Mon Jul 25 05:44:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:52.182441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:52.271837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:52.281804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27dd09-3574-461e-b0de-f419c924755c None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:52.314717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febc276-38c6-43eb-8b67-685914becabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:52.315356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc276-38c6-43eb-8b67-685914becabd None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:52.315641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc276-38c6-43eb-8b67-685914becabd None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:52.315918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6febc276-38c6-43eb-8b67-685914becabd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:52.382172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febc276-38c6-43eb-8b67-685914becabd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:52.383646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6febc276-38c6-43eb-8b67-685914becabd None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:52.469895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:52.479504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 640/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Mon Jul 25 05:44:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 25 05:44:52.485838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:52.546506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:52.584708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95791590-3b83-4255-8dc0-54b1e2178b0d None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:52.616750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:52.617532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:52.617813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:52.618084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:52.629075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:52.630523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 25 05:44:52.649536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:52.650623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:52.651896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 25 05:44:52.651896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 25 05:44:52.651896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 25 05:44:52.651896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:52.651896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 643/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:52.657079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febc276-38c6-43eb-8b67-685914becabd None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:52.753677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6febc276-38c6-43eb-8b67-685914becabd None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:52.788140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:52.788809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:52.789094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:52.789372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:52.790086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:52.790394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:52.791753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:52.803287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 641/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Mon Jul 25 05:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:52.809128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:52.907083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4a9a5-cb04-439b-8097-d8f09e3a8da4 None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:52.921062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:52.940782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:52.941562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:52.941847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:52.942125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:52.948516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:52.949990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 25 05:44:52.963907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:52.970698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: ERROR keystone.server.flask.application Jul 25 05:44:52.971764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 644/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Mon Jul 25 05:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:52.972410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:53.021329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:53.070818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10312f5f-0081-4add-8b4a-c629df2257c0 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:53.103427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8b71a-33be-43d7-b2bb-868a0ed782c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:53.104105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b71a-33be-43d7-b2bb-868a0ed782c9 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:53.104420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b71a-33be-43d7-b2bb-868a0ed782c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:53.104699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8b71a-33be-43d7-b2bb-868a0ed782c9 None admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:53.105480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b71a-33be-43d7-b2bb-868a0ed782c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:53.105794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b71a-33be-43d7-b2bb-868a0ed782c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:53.107762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8b71a-33be-43d7-b2bb-868a0ed782c9 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:53.119538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 642/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Mon Jul 25 05:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:53.126803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:53.227189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7489e0-5844-448f-8597-91b7ebf71a6b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:53.260722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd98667-1333-41be-beab-1c6ada74d969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:53.261372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98667-1333-41be-beab-1c6ada74d969 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:53.261655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98667-1333-41be-beab-1c6ada74d969 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:53.261936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd98667-1333-41be-beab-1c6ada74d969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:53.263129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd98667-1333-41be-beab-1c6ada74d969 None admin] RBAC: Adding query filter params (role.id=e9f347a638824dd09ef0cc73e9bb2e5b, scope.project.id=ca1bb14e085e4c8ab908e3880a657645, user.id=7c4988dc5513476786f11d08059c3d13) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:53.263447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd98667-1333-41be-beab-1c6ada74d969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:53.264916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd98667-1333-41be-beab-1c6ada74d969 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:53.270540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 645/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Mon Jul 25 05:44:53 2022] GET /identity/v3/role_assignments?user.id=7c4988dc5513476786f11d08059c3d13&scope.project.id=ca1bb14e085e4c8ab908e3880a657645&role.id=e9f347a638824dd09ef0cc73e9bb2e5b => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:55.360557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e4d-4a2e-47e7-b18f-4835e7e40719 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:55.361122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e4d-4a2e-47e7-b18f-4835e7e40719 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:55.361122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98557e4d-4a2e-47e7-b18f-4835e7e40719 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:55.362474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 643/1288] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:55.367728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c21451-242d-4b2b-9169-2b6f4d8af55d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:55.368179 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c21451-242d-4b2b-9169-2b6f4d8af55d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:55.368318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c21451-242d-4b2b-9169-2b6f4d8af55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:55.385140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f3c21451-242d-4b2b-9169-2b6f4d8af55d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:55.386365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f3c21451-242d-4b2b-9169-2b6f4d8af55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:55.416333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:55.453856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 646/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:55.476182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98557e4d-4a2e-47e7-b18f-4835e7e40719 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:55.581845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98557e4d-4a2e-47e7-b18f-4835e7e40719 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:55.593113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:55.615770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:55.616570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:55.616852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:55.622272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] PATH_INFO: `/v3/services/placement` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:55.622272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:55.622272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:44:55.631778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:44:55.633030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:44:55.634345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 644/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:44:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:44:55.639635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c21451-242d-4b2b-9169-2b6f4d8af55d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:55.738657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c21451-242d-4b2b-9169-2b6f4d8af55d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:55.772627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d7221-12be-40de-96eb-d818ab7a77b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:55.773297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d7221-12be-40de-96eb-d818ab7a77b8 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:55.773579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d7221-12be-40de-96eb-d818ab7a77b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:55.773854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d7221-12be-40de-96eb-d818ab7a77b8 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:55.774561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d7221-12be-40de-96eb-d818ab7a77b8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:55.774868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d7221-12be-40de-96eb-d818ab7a77b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:55.776286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d7221-12be-40de-96eb-d818ab7a77b8 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:55.781105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 647/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Jul 25 05:44:55 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:55.787019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:55.886547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b01c3b-3775-44a1-a662-6c21a34cff5f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:55.918111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f6fa6-83e9-4a00-abb9-55104d63dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:55.919392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f6fa6-83e9-4a00-abb9-55104d63dd29 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:55.919667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f6fa6-83e9-4a00-abb9-55104d63dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:55.919937 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f6fa6-83e9-4a00-abb9-55104d63dd29 None admin] PATH_INFO: `/v3/services` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:55.920665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f6fa6-83e9-4a00-abb9-55104d63dd29 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:44:55.920970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f6fa6-83e9-4a00-abb9-55104d63dd29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:55.922376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f6fa6-83e9-4a00-abb9-55104d63dd29 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:55.926715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 645/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Jul 25 05:44:55 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:44:57.941772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de39c3-06eb-4443-a48e-b3cf84601125 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.942343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de39c3-06eb-4443-a48e-b3cf84601125 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.942343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de39c3-06eb-4443-a48e-b3cf84601125 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.943704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 648/1293] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:44:57.948944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4131f-0099-4c97-b263-2aa47bae2fe0 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:57.949410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4131f-0099-4c97-b263-2aa47bae2fe0 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:57.949504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed4131f-0099-4c97-b263-2aa47bae2fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:57.966345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8ed4131f-0099-4c97-b263-2aa47bae2fe0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:44:57.967520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8ed4131f-0099-4c97-b263-2aa47bae2fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:44:57.995787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:58.032073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 646/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:44:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:44:58.054917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de39c3-06eb-4443-a48e-b3cf84601125 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:44:58.086341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:44:58.153704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de39c3-06eb-4443-a48e-b3cf84601125 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:44:58.188282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b2bae-534f-4723-9cfa-1cf6a5da8439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:44:58.188944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2bae-534f-4723-9cfa-1cf6a5da8439 None admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:44:58.189227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2bae-534f-4723-9cfa-1cf6a5da8439 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:44:58.189501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b2bae-534f-4723-9cfa-1cf6a5da8439 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:44:58.190231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b2bae-534f-4723-9cfa-1cf6a5da8439 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:44:58.191597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b2bae-534f-4723-9cfa-1cf6a5da8439 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:44:58.285854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 649/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Mon Jul 25 05:44:58 2022] POST /identity/v3/services => generated 246 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:45:00.373789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b054b3-c9df-40e3-a133-a6722759b8e6 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:00.374343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b054b3-c9df-40e3-a133-a6722759b8e6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:00.374343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b054b3-c9df-40e3-a133-a6722759b8e6 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:00.375744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 647/1296] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:45:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:45:00.381129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d791c-5882-47f8-a949-d0437335fcc4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:00.381536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d791c-5882-47f8-a949-d0437335fcc4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:00.381664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d791c-5882-47f8-a949-d0437335fcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:00.398220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-206d791c-5882-47f8-a949-d0437335fcc4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:00.399409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-206d791c-5882-47f8-a949-d0437335fcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:00.469905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 650/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:45:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:45:00.493234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b054b3-c9df-40e3-a133-a6722759b8e6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:00.593387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b054b3-c9df-40e3-a133-a6722759b8e6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:00.627468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:00.628273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:00.628556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:00.628832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] PATH_INFO: `/v3/services/placement` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:00.629478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:00.630842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:45:00.642528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:45:00.643698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:45:00.644787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 648/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:45:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 25 05:45:00.649870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d791c-5882-47f8-a949-d0437335fcc4 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:00.668066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:00.747527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d791c-5882-47f8-a949-d0437335fcc4 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:00.781648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:00.782345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:00.782631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:00.782909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:00.783621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] RBAC: Adding query filter params (name=placement) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:45:00.783931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:00.785345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:00.790511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 651/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Jul 25 05:45:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:45:00.797171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:00.828145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:00.894516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4431eb-2684-4a3c-b32f-7d83f439f07f None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:00.928656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10643890-77f4-47dc-b749-7cb7c493f34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:00.929310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643890-77f4-47dc-b749-7cb7c493f34c None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:00.929593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643890-77f4-47dc-b749-7cb7c493f34c None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:00.929864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10643890-77f4-47dc-b749-7cb7c493f34c None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:00.930644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10643890-77f4-47dc-b749-7cb7c493f34c None admin] RBAC: Adding query filter params (interface=public, service_id=421e6b13b8994a168932d0c5e4e95286, region_id=RegionOne) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:45:00.930954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10643890-77f4-47dc-b749-7cb7c493f34c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:00.932331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10643890-77f4-47dc-b749-7cb7c493f34c None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:00.937978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 649/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Mon Jul 25 05:45:00 2022] GET /identity/v3/endpoints?service_id=421e6b13b8994a168932d0c5e4e95286&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:45:00.943659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:01.024854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:01.042758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081d60f-8ffa-4a9e-85e0-89f8794aae6b None admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:01.076962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3b2c9-871f-4714-a657-b1388f0f5b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:01.077626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b2c9-871f-4714-a657-b1388f0f5b09 None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:01.077908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b2c9-871f-4714-a657-b1388f0f5b09 None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:01.078182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3b2c9-871f-4714-a657-b1388f0f5b09 None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:01.078864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3b2c9-871f-4714-a657-b1388f0f5b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:01.080282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3b2c9-871f-4714-a657-b1388f0f5b09 None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:01.086974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 652/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Mon Jul 25 05:45:00 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:45:03.118051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ccfea-a374-4432-881b-50be05bb85c6 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:03.118576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ccfea-a374-4432-881b-50be05bb85c6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:03.118576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ccfea-a374-4432-881b-50be05bb85c6 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:03.119875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 650/1302] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:45:03.125713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f785540-f41e-45f1-be58-8470b794396d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:03.126130 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f785540-f41e-45f1-be58-8470b794396d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:03.126273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f785540-f41e-45f1-be58-8470b794396d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:03.143383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9f785540-f41e-45f1-be58-8470b794396d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:45:03.144617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9f785540-f41e-45f1-be58-8470b794396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:45:03.210393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 653/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:45:03 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 25 05:45:03.232713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ccfea-a374-4432-881b-50be05bb85c6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:03.326708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:03.333212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ccfea-a374-4432-881b-50be05bb85c6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:03.366276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:03.367072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:03.367358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:03.367638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:03.368401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:03.369743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 25 05:45:03.382926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 25 05:45:03.384142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.get_or_create( Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application with Lock( Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self._enter() Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application return self.creator() Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application created_value = creator( Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: ERROR keystone.server.flask.application Jul 25 05:45:03.385195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 651/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Mon Jul 25 05:45:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 25 05:45:03.390337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f785540-f41e-45f1-be58-8470b794396d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:03.465714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:03.488007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f785540-f41e-45f1-be58-8470b794396d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:03.520944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f75eb0-f541-47aa-aab2-f034b5de42bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:03.521600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75eb0-f541-47aa-aab2-f034b5de42bc None admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:03.521921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75eb0-f541-47aa-aab2-f034b5de42bc None admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:03.522260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f75eb0-f541-47aa-aab2-f034b5de42bc None admin] PATH_INFO: `/v3/services` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:03.522969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f75eb0-f541-47aa-aab2-f034b5de42bc None admin] RBAC: Adding query filter params (name=placement) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:45:03.523278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f75eb0-f541-47aa-aab2-f034b5de42bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:03.524767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f75eb0-f541-47aa-aab2-f034b5de42bc None admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:03.533383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 654/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Mon Jul 25 05:45:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:45:03.535700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:45:03.633677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c66b20-77a1-463d-9b94-2a87562194f5 None admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:45:03.665986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c435b6-7504-4216-9745-05bf6ea7d176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:45:03.666636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c435b6-7504-4216-9745-05bf6ea7d176 None admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:45:03.666915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c435b6-7504-4216-9745-05bf6ea7d176 None admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:45:03.667188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c435b6-7504-4216-9745-05bf6ea7d176 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:45:03.667930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c435b6-7504-4216-9745-05bf6ea7d176 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:45:03.669984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c435b6-7504-4216-9745-05bf6ea7d176 None admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:45:03.823077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 652/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Mon Jul 25 05:45:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:45:13.531337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=30 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:45:13.818220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:50:59.907703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04af96-5383-4715-ac01-83044ed7233d None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:59.908316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04af96-5383-4715-ac01-83044ed7233d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:59.908316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b04af96-5383-4715-ac01-83044ed7233d None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:59.915836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636568-02d8-4d7e-bd3d-f74838ce7492 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:50:59.916334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636568-02d8-4d7e-bd3d-f74838ce7492 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:50:59.916437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9636568-02d8-4d7e-bd3d-f74838ce7492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:50:59.918343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 655/1307] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:50:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:00.005602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f9636568-02d8-4d7e-bd3d-f74838ce7492 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:00.006970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f9636568-02d8-4d7e-bd3d-f74838ce7492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:00.130067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 653/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:50:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:00.188495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38a358-8cb5-4c47-aa0a-ee6d16a45a76 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:00.195373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38a358-8cb5-4c47-aa0a-ee6d16a45a76 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:00.195373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38a358-8cb5-4c47-aa0a-ee6d16a45a76 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:00.195373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 656/1309] 10.222.0.40 () {54 vars in 965 bytes} [Mon Jul 25 05:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:00.199578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee1798-1763-4d1f-ba28-e99d9b338b2e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:00.199871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee1798-1763-4d1f-ba28-e99d9b338b2e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:00.200165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee1798-1763-4d1f-ba28-e99d9b338b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:00.284703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e6ee1798-1763-4d1f-ba28-e99d9b338b2e None None] MFA Rules not processed for user `b1638c28af8240439d061a3ae0ddc38d`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:00.285914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e6ee1798-1763-4d1f-ba28-e99d9b338b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:00.318111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:00.381561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 654/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Mon Jul 25 05:51:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:00.395334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38a358-8cb5-4c47-aa0a-ee6d16a45a76 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:00.399349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9a38a358-8cb5-4c47-aa0a-ee6d16a45a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:00.552868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38a358-8cb5-4c47-aa0a-ee6d16a45a76 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:00.638445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59796317-8474-405a-a822-58e82a5e002a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:00.640103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796317-8474-405a-a822-58e82a5e002a service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:00.640495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796317-8474-405a-a822-58e82a5e002a service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:00.640853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59796317-8474-405a-a822-58e82a5e002a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:00.642607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-59796317-8474-405a-a822-58e82a5e002a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:00.702483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:00.769296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59796317-8474-405a-a822-58e82a5e002a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:00.770916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59796317-8474-405a-a822-58e82a5e002a service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:00.791791 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 657/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:03.154698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc8a1-99dd-408c-8bb6-fc3db3611b46 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:03.155229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc8a1-99dd-408c-8bb6-fc3db3611b46 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:03.155229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc8a1-99dd-408c-8bb6-fc3db3611b46 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:03.156570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 655/1312] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:03.161636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdedf65-a32b-402c-9db3-b1115f2535e9 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:03.162063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdedf65-a32b-402c-9db3-b1115f2535e9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:03.162226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdedf65-a32b-402c-9db3-b1115f2535e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:03.234380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2fdedf65-a32b-402c-9db3-b1115f2535e9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:03.235624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2fdedf65-a32b-402c-9db3-b1115f2535e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:03.333081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 658/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:03.372961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dc8a1-99dd-408c-8bb6-fc3db3611b46 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:03.374541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f02dc8a1-99dd-408c-8bb6-fc3db3611b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:03.486441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dc8a1-99dd-408c-8bb6-fc3db3611b46 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:03.560615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:03.562910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d66d610-bbf3-4f09-86c1-7b16e00663e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:03.563566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66d610-bbf3-4f09-86c1-7b16e00663e8 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:03.563851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66d610-bbf3-4f09-86c1-7b16e00663e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:03.564156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66d610-bbf3-4f09-86c1-7b16e00663e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:03.565824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0d66d610-bbf3-4f09-86c1-7b16e00663e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:03.676505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66d610-bbf3-4f09-86c1-7b16e00663e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:03.677967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d66d610-bbf3-4f09-86c1-7b16e00663e8 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:03.696409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 656/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:06.265571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285598ee-7ca1-42c6-89c6-f6e60a4b815e None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:06.266131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285598ee-7ca1-42c6-89c6-f6e60a4b815e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:06.266131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285598ee-7ca1-42c6-89c6-f6e60a4b815e None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:06.267502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 659/1315] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:06.272780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148df5f-d1ac-4fee-984d-e232b63760ee None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:06.273196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148df5f-d1ac-4fee-984d-e232b63760ee None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:06.273338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c148df5f-d1ac-4fee-984d-e232b63760ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:06.338142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c148df5f-d1ac-4fee-984d-e232b63760ee None None] MFA Rules not processed for user `388e066787d64c8f80e53b8373170ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:06.339373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c148df5f-d1ac-4fee-984d-e232b63760ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:06.391564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:06.436018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 657/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:06.476921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285598ee-7ca1-42c6-89c6-f6e60a4b815e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:06.478621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-285598ee-7ca1-42c6-89c6-f6e60a4b815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:06.488438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:06.586431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285598ee-7ca1-42c6-89c6-f6e60a4b815e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:06.658160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b11a3-5d6f-48a4-a9e6-a83c2d162634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:06.658810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b11a3-5d6f-48a4-a9e6-a83c2d162634 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:06.659101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b11a3-5d6f-48a4-a9e6-a83c2d162634 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:06.659407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b11a3-5d6f-48a4-a9e6-a83c2d162634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:06.661108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e81b11a3-5d6f-48a4-a9e6-a83c2d162634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:06.722753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:06.770458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b11a3-5d6f-48a4-a9e6-a83c2d162634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:06.771960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b11a3-5d6f-48a4-a9e6-a83c2d162634 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:06.792417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 660/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:09.629787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd188160-6dd2-4a9a-a334-b2267704ccb2 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:09.630371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd188160-6dd2-4a9a-a334-b2267704ccb2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:09.630371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd188160-6dd2-4a9a-a334-b2267704ccb2 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:09.632300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 658/1318] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:09.637463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe829fd-41ca-42e8-ba9b-70107fd4f3ea None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:09.637847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe829fd-41ca-42e8-ba9b-70107fd4f3ea None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:09.638027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe829fd-41ca-42e8-ba9b-70107fd4f3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:09.701434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-bfe829fd-41ca-42e8-ba9b-70107fd4f3ea None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:09.702647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bfe829fd-41ca-42e8-ba9b-70107fd4f3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:09.798843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 661/1319] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:09.838656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd188160-6dd2-4a9a-a334-b2267704ccb2 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:09.840230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fd188160-6dd2-4a9a-a334-b2267704ccb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:09.941798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd188160-6dd2-4a9a-a334-b2267704ccb2 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:10.012942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155c0bb-86f8-4746-9bbd-922c0f662fcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:10.013588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155c0bb-86f8-4746-9bbd-922c0f662fcf service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:10.013872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155c0bb-86f8-4746-9bbd-922c0f662fcf service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:10.014151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155c0bb-86f8-4746-9bbd-922c0f662fcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:10.015802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a155c0bb-86f8-4746-9bbd-922c0f662fcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:10.023194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:10.121507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155c0bb-86f8-4746-9bbd-922c0f662fcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:10.122946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155c0bb-86f8-4746-9bbd-922c0f662fcf service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:10.141074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 659/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:13.181870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e39c1a-95b2-4c8b-89c8-c94aba17e53b None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:13.182401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e39c1a-95b2-4c8b-89c8-c94aba17e53b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:13.182401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e39c1a-95b2-4c8b-89c8-c94aba17e53b None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:13.183801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 662/1321] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:13.188965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a7f41-ae06-4f02-a32b-6317e94ab023 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:13.189353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a7f41-ae06-4f02-a32b-6317e94ab023 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:13.189530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a7f41-ae06-4f02-a32b-6317e94ab023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:13.252882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9a7a7f41-ae06-4f02-a32b-6317e94ab023 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:13.254082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9a7a7f41-ae06-4f02-a32b-6317e94ab023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:13.311070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:13.349327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 660/1322] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:13.385480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e39c1a-95b2-4c8b-89c8-c94aba17e53b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:13.387075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-06e39c1a-95b2-4c8b-89c8-c94aba17e53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:13.408912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:13.503018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e39c1a-95b2-4c8b-89c8-c94aba17e53b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:13.582949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2474a-422e-4d01-926e-30052bba2520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:13.583679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2474a-422e-4d01-926e-30052bba2520 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:13.583959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2474a-422e-4d01-926e-30052bba2520 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:13.584808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2474a-422e-4d01-926e-30052bba2520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:13.586761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-27c2474a-422e-4d01-926e-30052bba2520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:13.660613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:13.700436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2474a-422e-4d01-926e-30052bba2520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:13.701981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2474a-422e-4d01-926e-30052bba2520 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:13.719866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 663/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:15.832398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929164-63c8-407e-9e73-4cbbafca0e68 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:15.832947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929164-63c8-407e-9e73-4cbbafca0e68 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:15.833039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929164-63c8-407e-9e73-4cbbafca0e68 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:15.834873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 661/1324] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:15.840249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ab76c-0a77-41e2-9371-cacfba6540e1 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:15.840680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ab76c-0a77-41e2-9371-cacfba6540e1 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:15.840822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ab76c-0a77-41e2-9371-cacfba6540e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:15.906665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-137ab76c-0a77-41e2-9371-cacfba6540e1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:15.907795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-137ab76c-0a77-41e2-9371-cacfba6540e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:16.004816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 664/1325] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:16.041539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f929164-63c8-407e-9e73-4cbbafca0e68 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:16.043099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3f929164-63c8-407e-9e73-4cbbafca0e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:16.143995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f929164-63c8-407e-9e73-4cbbafca0e68 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:16.218146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9663e-b403-4949-9ad1-9b624fbd1a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:16.218801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9663e-b403-4949-9ad1-9b624fbd1a79 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:16.219090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9663e-b403-4949-9ad1-9b624fbd1a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:16.219368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9663e-b403-4949-9ad1-9b624fbd1a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:16.221061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e8d9663e-b403-4949-9ad1-9b624fbd1a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:16.238463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:16.336578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9663e-b403-4949-9ad1-9b624fbd1a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:16.338038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d9663e-b403-4949-9ad1-9b624fbd1a79 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:16.356959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 662/1326] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:18.467804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b788f-2fed-4aa3-9590-7234c6e94ef2 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:18.468366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b788f-2fed-4aa3-9590-7234c6e94ef2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:18.468484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b788f-2fed-4aa3-9590-7234c6e94ef2 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:18.469790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 665/1327] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:18.476077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7dd69-caf9-432b-b80b-2bfd1fced4d7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:18.476553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7dd69-caf9-432b-b80b-2bfd1fced4d7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:18.487375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7dd69-caf9-432b-b80b-2bfd1fced4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:18.542809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b8c7dd69-caf9-432b-b80b-2bfd1fced4d7 None None] MFA Rules not processed for user `388e066787d64c8f80e53b8373170ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:18.544031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b8c7dd69-caf9-432b-b80b-2bfd1fced4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:18.612319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:18.645601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 663/1328] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:18 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:18.698301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12affd35-898c-47cd-a4fe-85a766a15d49 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:18.698595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12affd35-898c-47cd-a4fe-85a766a15d49 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:18.698877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12affd35-898c-47cd-a4fe-85a766a15d49 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:18.700337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 666/1329] 10.222.0.40 () {54 vars in 965 bytes} [Mon Jul 25 05:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:18.706870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa31a-9212-49f4-8092-ae7e70f217d0 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:18.707160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa31a-9212-49f4-8092-ae7e70f217d0 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:18.707438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa31a-9212-49f4-8092-ae7e70f217d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:18.784593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-762fa31a-9212-49f4-8092-ae7e70f217d0 None None] MFA Rules not processed for user `b1638c28af8240439d061a3ae0ddc38d`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:18.785798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-762fa31a-9212-49f4-8092-ae7e70f217d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:18.863990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 664/1330] 10.222.0.40 () {58 vars in 1062 bytes} [Mon Jul 25 05:51:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:18.873518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12affd35-898c-47cd-a4fe-85a766a15d49 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:18.875044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-12affd35-898c-47cd-a4fe-85a766a15d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:18.909348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:18.983650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12affd35-898c-47cd-a4fe-85a766a15d49 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:19.056988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c74232-0463-4cfa-b8cc-e32fe2c35d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:19.057638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c74232-0463-4cfa-b8cc-e32fe2c35d6b service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:19.057921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c74232-0463-4cfa-b8cc-e32fe2c35d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:19.058195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c74232-0463-4cfa-b8cc-e32fe2c35d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:19.059853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d0c74232-0463-4cfa-b8cc-e32fe2c35d6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:19.158471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:19.194038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c74232-0463-4cfa-b8cc-e32fe2c35d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:19.195898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c74232-0463-4cfa-b8cc-e32fe2c35d6b service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:19.214080 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 667/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:22.156501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce823-10f3-42f5-9ca1-7ed379661f66 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:22.157114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce823-10f3-42f5-9ca1-7ed379661f66 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:22.157114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ce823-10f3-42f5-9ca1-7ed379661f66 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:22.158440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 665/1332] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:22.164341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36766c7-8621-4a83-af60-321adb4b86e3 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:22.164792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36766c7-8621-4a83-af60-321adb4b86e3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:22.164927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36766c7-8621-4a83-af60-321adb4b86e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:22.229124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c36766c7-8621-4a83-af60-321adb4b86e3 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:22.230323 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c36766c7-8621-4a83-af60-321adb4b86e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:22.325960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 668/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:22.361910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ce823-10f3-42f5-9ca1-7ed379661f66 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:22.363466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-989ce823-10f3-42f5-9ca1-7ed379661f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:22.432494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:22.467172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ce823-10f3-42f5-9ca1-7ed379661f66 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:22.546296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7939eccb-3b4b-47fb-b222-f62dc182a476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:22.546941 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939eccb-3b4b-47fb-b222-f62dc182a476 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:22.547223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939eccb-3b4b-47fb-b222-f62dc182a476 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:22.547498 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939eccb-3b4b-47fb-b222-f62dc182a476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:22.549716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7939eccb-3b4b-47fb-b222-f62dc182a476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:22.662202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939eccb-3b4b-47fb-b222-f62dc182a476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:22.663577 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7939eccb-3b4b-47fb-b222-f62dc182a476 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:22.673271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:22.688218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 666/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:24.766030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8bfb5-66c9-4566-9b2c-444ec3e6201b None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:24.766584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8bfb5-66c9-4566-9b2c-444ec3e6201b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:24.766584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c8bfb5-66c9-4566-9b2c-444ec3e6201b None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:24.768360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 669/1335] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:24.773897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd849bb-8301-4821-bb47-d0880c423939 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:24.774297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd849bb-8301-4821-bb47-d0880c423939 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:24.774454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd849bb-8301-4821-bb47-d0880c423939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:24.837102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-bfd849bb-8301-4821-bb47-d0880c423939 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:24.838289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bfd849bb-8301-4821-bb47-d0880c423939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:24.931105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 667/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:24.970628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8bfb5-66c9-4566-9b2c-444ec3e6201b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:24.972272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-76c8bfb5-66c9-4566-9b2c-444ec3e6201b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:25.011963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:25.074211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c8bfb5-66c9-4566-9b2c-444ec3e6201b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:25.145180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6bb9e-d279-4c70-a469-5184ec569179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:25.145827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6bb9e-d279-4c70-a469-5184ec569179 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:25.146146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6bb9e-d279-4c70-a469-5184ec569179 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:25.146502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6bb9e-d279-4c70-a469-5184ec569179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:25.148200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d0a6bb9e-d279-4c70-a469-5184ec569179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:25.230061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:25.255520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6bb9e-d279-4c70-a469-5184ec569179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:25.257074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6bb9e-d279-4c70-a469-5184ec569179 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:25.275334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 670/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:28.031799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414443-d197-401e-bd8a-248649554c66 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.032365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414443-d197-401e-bd8a-248649554c66 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.032365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4414443-d197-401e-bd8a-248649554c66 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.033677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 668/1338] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:28.039569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde24e-31e9-43b6-814d-c62f067d2851 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.039977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde24e-31e9-43b6-814d-c62f067d2851 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.040173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dde24e-31e9-43b6-814d-c62f067d2851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.103827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f2dde24e-31e9-43b6-814d-c62f067d2851 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:28.105070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f2dde24e-31e9-43b6-814d-c62f067d2851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:28.199948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 671/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:28.236435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4414443-d197-401e-bd8a-248649554c66 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:28.237995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e4414443-d197-401e-bd8a-248649554c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:28.314886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:28.343045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4414443-d197-401e-bd8a-248649554c66 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:28.416002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea4cc3-29ea-412e-8f5e-80d28765bfb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:28.416670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4cc3-29ea-412e-8f5e-80d28765bfb2 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:28.416950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4cc3-29ea-412e-8f5e-80d28765bfb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:28.417225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea4cc3-29ea-412e-8f5e-80d28765bfb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:28.418854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b0ea4cc3-29ea-412e-8f5e-80d28765bfb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:28.527154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea4cc3-29ea-412e-8f5e-80d28765bfb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:28.528616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea4cc3-29ea-412e-8f5e-80d28765bfb2 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:28.547079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 669/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:32.836501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dedfab-7268-4365-9209-794dc033973c None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:32.837006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dedfab-7268-4365-9209-794dc033973c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:32.837132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dedfab-7268-4365-9209-794dc033973c None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:32.838585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 672/1341] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:32.843710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa61b2-48c2-4dbb-9cdb-5cf2ea98b2c6 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:32.844281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa61b2-48c2-4dbb-9cdb-5cf2ea98b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:32.844281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa61b2-48c2-4dbb-9cdb-5cf2ea98b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:32.848455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:32.916757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7daa61b2-48c2-4dbb-9cdb-5cf2ea98b2c6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:32.917590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7daa61b2-48c2-4dbb-9cdb-5cf2ea98b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:33.017481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 670/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:33.061410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dedfab-7268-4365-9209-794dc033973c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:33.063076 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-91dedfab-7268-4365-9209-794dc033973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:33.120886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:33.178638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dedfab-7268-4365-9209-794dc033973c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:33.251966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ca14c-c0f8-4f4e-b8e7-b20878d6f0cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:33.252646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca14c-c0f8-4f4e-b8e7-b20878d6f0cd service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:33.252927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca14c-c0f8-4f4e-b8e7-b20878d6f0cd service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:33.253201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca14c-c0f8-4f4e-b8e7-b20878d6f0cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:33.254861 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bf6ca14c-c0f8-4f4e-b8e7-b20878d6f0cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:33.345930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:33.362647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ca14c-c0f8-4f4e-b8e7-b20878d6f0cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:33.364095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ca14c-c0f8-4f4e-b8e7-b20878d6f0cd service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:33.383169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 673/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:36.463929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43ea4-904e-42ca-b35d-3d3b55609825 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:36.464490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43ea4-904e-42ca-b35d-3d3b55609825 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:36.464621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43ea4-904e-42ca-b35d-3d3b55609825 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:36.465943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 671/1344] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:36.471236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d81670-2986-4eb1-8aac-6e6e646a1a5a None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:36.471659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d81670-2986-4eb1-8aac-6e6e646a1a5a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:36.471799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d81670-2986-4eb1-8aac-6e6e646a1a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:36.537187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d4d81670-2986-4eb1-8aac-6e6e646a1a5a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:36.538378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d4d81670-2986-4eb1-8aac-6e6e646a1a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:36.635352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 674/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:36.673168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43ea4-904e-42ca-b35d-3d3b55609825 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:36.674987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-efc43ea4-904e-42ca-b35d-3d3b55609825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:36.764869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:36.783356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43ea4-904e-42ca-b35d-3d3b55609825 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:36.978204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2c9d0-07cb-4fd7-b598-be80029c9655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:36.978879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2c9d0-07cb-4fd7-b598-be80029c9655 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:36.979168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2c9d0-07cb-4fd7-b598-be80029c9655 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:36.979453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2c9d0-07cb-4fd7-b598-be80029c9655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:36.981244 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f8f2c9d0-07cb-4fd7-b598-be80029c9655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:37.094478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2c9d0-07cb-4fd7-b598-be80029c9655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:37.095920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2c9d0-07cb-4fd7-b598-be80029c9655 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:37.114874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 672/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:41.607039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfd976-d253-4646-8afa-01d016236168 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:41.607534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfd976-d253-4646-8afa-01d016236168 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:41.607618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfd976-d253-4646-8afa-01d016236168 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:41.609064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 675/1347] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:41.615026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a123ff-02c5-4812-807b-61808854f01c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:41.615495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a123ff-02c5-4812-807b-61808854f01c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:41.615740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a123ff-02c5-4812-807b-61808854f01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:41.631538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:41.687386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e2a123ff-02c5-4812-807b-61808854f01c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:41.688618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e2a123ff-02c5-4812-807b-61808854f01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:41.796508 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 673/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:41.829055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfd976-d253-4646-8afa-01d016236168 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:41.830613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8cbfd976-d253-4646-8afa-01d016236168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:41.892275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:41.940686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfd976-d253-4646-8afa-01d016236168 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:42.015960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e05442-0713-42bd-a5cc-1b71c458582e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:42.016648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e05442-0713-42bd-a5cc-1b71c458582e service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:42.016933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e05442-0713-42bd-a5cc-1b71c458582e service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:42.017213 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e05442-0713-42bd-a5cc-1b71c458582e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:42.018897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d1e05442-0713-42bd-a5cc-1b71c458582e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:42.130361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:42.134743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e05442-0713-42bd-a5cc-1b71c458582e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:42.136312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e05442-0713-42bd-a5cc-1b71c458582e service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:42.155271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 676/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:47.055084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7202c-d942-474a-b244-bcc0a5573051 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:47.057464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7202c-d942-474a-b244-bcc0a5573051 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:47.057754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7202c-d942-474a-b244-bcc0a5573051 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:47.059197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 674/1350] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 05:51:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:47.064140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f874c2-70c2-423a-a704-01ba09302254 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:47.064597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f874c2-70c2-423a-a704-01ba09302254 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:47.064725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f874c2-70c2-423a-a704-01ba09302254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:47.154996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-18f874c2-70c2-423a-a704-01ba09302254 None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:47.156512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-18f874c2-70c2-423a-a704-01ba09302254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:47.294301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 677/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 05:51:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:51.455859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a305973-7bb7-4087-98b7-fad668f8f838 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:51.456465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a305973-7bb7-4087-98b7-fad668f8f838 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:51.456465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a305973-7bb7-4087-98b7-fad668f8f838 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:51.458374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 675/1352] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 05:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:51.463515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e7918-6c65-4367-b9f7-ba636a467d67 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:51.464398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e7918-6c65-4367-b9f7-ba636a467d67 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:51.464885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e7918-6c65-4367-b9f7-ba636a467d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:51.523246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:51.551518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c87e7918-6c65-4367-b9f7-ba636a467d67 None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:51.553121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c87e7918-6c65-4367-b9f7-ba636a467d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:51.682433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 678/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 05:51:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:51.788257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=59 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:52.857590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3bd33-7f32-4c20-92ee-46eb2359bf7b None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:52.887698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3bd33-7f32-4c20-92ee-46eb2359bf7b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:52.887698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3bd33-7f32-4c20-92ee-46eb2359bf7b None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:52.887698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 676/1354] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 05:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:52.887985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ce78c-a79e-4d38-aa2d-7bda875481d8 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:52.887985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ce78c-a79e-4d38-aa2d-7bda875481d8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:52.887985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ce78c-a79e-4d38-aa2d-7bda875481d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:52.944560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0d4ce78c-a79e-4d38-aa2d-7bda875481d8 None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:52.945750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0d4ce78c-a79e-4d38-aa2d-7bda875481d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:52.990309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:53.056294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 679/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 05:51:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:56.640515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251066c7-f492-4eba-b4d9-48e6f6a426b3 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:56.641390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251066c7-f492-4eba-b4d9-48e6f6a426b3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:56.642004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251066c7-f492-4eba-b4d9-48e6f6a426b3 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:56.643889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 677/1356] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:56.650736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fa361-7cd7-43e4-a7ad-439b95e3837d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:56.651232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fa361-7cd7-43e4-a7ad-439b95e3837d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:56.651314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fa361-7cd7-43e4-a7ad-439b95e3837d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:56.722372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f70fa361-7cd7-43e4-a7ad-439b95e3837d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:56.723701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f70fa361-7cd7-43e4-a7ad-439b95e3837d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:56.837366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 680/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:56.893284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1effdad-0a4c-4026-b86d-a01d083bf9c4 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:56.893920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1effdad-0a4c-4026-b86d-a01d083bf9c4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:56.895041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1effdad-0a4c-4026-b86d-a01d083bf9c4 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:56.896744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 678/1358] 10.222.0.40 () {54 vars in 962 bytes} [Mon Jul 25 05:51:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:56.907810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d327edd-6b29-4ab7-852e-68f6c42393c9 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:56.913882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d327edd-6b29-4ab7-852e-68f6c42393c9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:56.913882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d327edd-6b29-4ab7-852e-68f6c42393c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:56.924910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:56.992993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0d327edd-6b29-4ab7-852e-68f6c42393c9 None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:56.994184 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0d327edd-6b29-4ab7-852e-68f6c42393c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:57.118911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 681/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 25 05:51:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:51:57.127006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1effdad-0a4c-4026-b86d-a01d083bf9c4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:51:57.129089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d1effdad-0a4c-4026-b86d-a01d083bf9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:57.283763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1effdad-0a4c-4026-b86d-a01d083bf9c4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:51:57.365564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:57.407546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4860179-0690-4730-bae8-108f14651b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:51:57.408403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860179-0690-4730-bae8-108f14651b40 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:57.408692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860179-0690-4730-bae8-108f14651b40 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:57.408968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4860179-0690-4730-bae8-108f14651b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:57.411564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e4860179-0690-4730-bae8-108f14651b40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:57.508340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860179-0690-4730-bae8-108f14651b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:51:57.509993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4860179-0690-4730-bae8-108f14651b40 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:51:57.530737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 679/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:51:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:51:59.787909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a386ba-0e8a-4b36-9cc6-be079fd4c781 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:59.788474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a386ba-0e8a-4b36-9cc6-be079fd4c781 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:59.788474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a386ba-0e8a-4b36-9cc6-be079fd4c781 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:59.790229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 682/1361] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:51:59.795825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2492-7c6e-4863-91aa-fb77ffcbbc4a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:51:59.796285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2492-7c6e-4863-91aa-fb77ffcbbc4a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:51:59.796479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2492-7c6e-4863-91aa-fb77ffcbbc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:51:59.866624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-494e2492-7c6e-4863-91aa-fb77ffcbbc4a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:51:59.868016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-494e2492-7c6e-4863-91aa-fb77ffcbbc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:51:59.906180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:51:59.968073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 680/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:00.012105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a386ba-0e8a-4b36-9cc6-be079fd4c781 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:00.013834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-12a386ba-0e8a-4b36-9cc6-be079fd4c781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:00.088162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:00.179124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a386ba-0e8a-4b36-9cc6-be079fd4c781 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:00.330851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e24bfa-b403-4cfa-89f5-ca1c2c910bba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:00.332389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e24bfa-b403-4cfa-89f5-ca1c2c910bba service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:00.332781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e24bfa-b403-4cfa-89f5-ca1c2c910bba service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:00.333277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e24bfa-b403-4cfa-89f5-ca1c2c910bba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:00.335027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-84e24bfa-b403-4cfa-89f5-ca1c2c910bba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:00.360090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:00.430176 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e24bfa-b403-4cfa-89f5-ca1c2c910bba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:00.431803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e24bfa-b403-4cfa-89f5-ca1c2c910bba service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:00.451883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 683/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:03.021633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da96b1-0cae-4da2-add2-fac7ba4ffb1e None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:03.022174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da96b1-0cae-4da2-add2-fac7ba4ffb1e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:03.022174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da96b1-0cae-4da2-add2-fac7ba4ffb1e None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:03.023624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 681/1364] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:03.028806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd436-8871-4c95-b6f0-974245f4dd34 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:03.029228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd436-8871-4c95-b6f0-974245f4dd34 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:03.029405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd436-8871-4c95-b6f0-974245f4dd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:03.098471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d7cbd436-8871-4c95-b6f0-974245f4dd34 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:03.099688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d7cbd436-8871-4c95-b6f0-974245f4dd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:03.201847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:03.208189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 684/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:03.257590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb35068-b400-4232-ac2b-61e31be5dea6 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:03.257882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb35068-b400-4232-ac2b-61e31be5dea6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:03.258399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb35068-b400-4232-ac2b-61e31be5dea6 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:03.260061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 682/1366] 10.222.0.40 () {54 vars in 962 bytes} [Mon Jul 25 05:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:03.267078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8a2fd-1b4b-4d68-9354-87196ee9e8c2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:03.267491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8a2fd-1b4b-4d68-9354-87196ee9e8c2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:03.267875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8a2fd-1b4b-4d68-9354-87196ee9e8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:03.352436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c4b8a2fd-1b4b-4d68-9354-87196ee9e8c2 None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:03.353904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c4b8a2fd-1b4b-4d68-9354-87196ee9e8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:03.479533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 685/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 25 05:52:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:03.487340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb35068-b400-4232-ac2b-61e31be5dea6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:03.489322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0fb35068-b400-4232-ac2b-61e31be5dea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:03.642395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb35068-b400-4232-ac2b-61e31be5dea6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:03.664613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:03.771048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859ec31-14c5-483d-9253-65ec8792dc75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:03.771724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859ec31-14c5-483d-9253-65ec8792dc75 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:03.772012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859ec31-14c5-483d-9253-65ec8792dc75 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:03.772336 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859ec31-14c5-483d-9253-65ec8792dc75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:03.774073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b859ec31-14c5-483d-9253-65ec8792dc75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:03.878675 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859ec31-14c5-483d-9253-65ec8792dc75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:03.880389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859ec31-14c5-483d-9253-65ec8792dc75 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:03.899677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 683/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:04.770252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ba554-9f0c-414c-a91d-c69522a0c41e None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:04.770252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ba554-9f0c-414c-a91d-c69522a0c41e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:04.770670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ba554-9f0c-414c-a91d-c69522a0c41e None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:04.771947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 686/1369] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:04.777235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e7124-9a34-4566-82c7-b7d6e194e055 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:04.777689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e7124-9a34-4566-82c7-b7d6e194e055 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:04.777768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e7124-9a34-4566-82c7-b7d6e194e055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:04.810099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:04.845527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-046e7124-9a34-4566-82c7-b7d6e194e055 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:04.847589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-046e7124-9a34-4566-82c7-b7d6e194e055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:04.954825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 684/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:05.305859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcd3de-d1c5-4192-9990-68a7a867acde None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:05.306152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcd3de-d1c5-4192-9990-68a7a867acde None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:05.306434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcd3de-d1c5-4192-9990-68a7a867acde None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:05.307956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 687/1371] 10.222.0.40 () {54 vars in 964 bytes} [Mon Jul 25 05:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:05.312999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6efbde-06be-4040-a4d9-8977539a7c5b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:05.313393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6efbde-06be-4040-a4d9-8977539a7c5b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:05.313773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6efbde-06be-4040-a4d9-8977539a7c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:05.394328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2c6efbde-06be-4040-a4d9-8977539a7c5b None None] MFA Rules not processed for user `f8a47b60be1a45f9aabc262b2d04d596`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:05.395679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2c6efbde-06be-4040-a4d9-8977539a7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:05.444497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:05.476362 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 685/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Mon Jul 25 05:52:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:52:05.483990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcd3de-d1c5-4192-9990-68a7a867acde None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:05.485621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-28fcd3de-d1c5-4192-9990-68a7a867acde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:05.544752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:05.601640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcd3de-d1c5-4192-9990-68a7a867acde None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:05.682304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b8269-3375-45e7-af19-9f635fdad56e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:05.683250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b8269-3375-45e7-af19-9f635fdad56e service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:05.683720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b8269-3375-45e7-af19-9f635fdad56e service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:05.684339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b8269-3375-45e7-af19-9f635fdad56e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:05.686175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bd3b8269-3375-45e7-af19-9f635fdad56e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:05.794256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:05.810716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b8269-3375-45e7-af19-9f635fdad56e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:05.812281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b8269-3375-45e7-af19-9f635fdad56e service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:05.831399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 688/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:06.157577 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8e83-4886-4089-b608-615dbbb6f14a None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:06.158080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8e83-4886-4089-b608-615dbbb6f14a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:06.158080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8e83-4886-4089-b608-615dbbb6f14a None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:06.159831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 686/1374] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:52:06.164912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff1f8-4634-4f4a-a53e-6b49ed2eec1d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:06.165133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff1f8-4634-4f4a-a53e-6b49ed2eec1d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:06.165431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ff1f8-4634-4f4a-a53e-6b49ed2eec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:06.230534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-337ff1f8-4634-4f4a-a53e-6b49ed2eec1d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:06.231767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-337ff1f8-4634-4f4a-a53e-6b49ed2eec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:06.332719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 689/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:06.376784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e8e83-4886-4089-b608-615dbbb6f14a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:06.378705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-057e8e83-4886-4089-b608-615dbbb6f14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:06.531910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e8e83-4886-4089-b608-615dbbb6f14a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:06.584024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:06.646302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc6bfd-31f3-475d-add0-5598c60a0a0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:06.646949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6bfd-31f3-475d-add0-5598c60a0a0b service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:06.647231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6bfd-31f3-475d-add0-5598c60a0a0b service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:06.647504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc6bfd-31f3-475d-add0-5598c60a0a0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:06.649183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-64dc6bfd-31f3-475d-add0-5598c60a0a0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:06.735831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc6bfd-31f3-475d-add0-5598c60a0a0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:06.737605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc6bfd-31f3-475d-add0-5598c60a0a0b service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:06.759210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 687/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:08.956554 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9306df7-d8bc-4b63-82b5-5667a7eac83f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:08.957237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9306df7-d8bc-4b63-82b5-5667a7eac83f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:08.957526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9306df7-d8bc-4b63-82b5-5667a7eac83f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:08.958917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 690/1377] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:08.964179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cf994-db00-473f-9335-0aa0bb6217cc None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:08.964590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cf994-db00-473f-9335-0aa0bb6217cc None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:08.964682 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69cf994-db00-473f-9335-0aa0bb6217cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:09.029684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f69cf994-db00-473f-9335-0aa0bb6217cc None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:09.030910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f69cf994-db00-473f-9335-0aa0bb6217cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:09.038455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:09.131944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 688/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:09.177115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9306df7-d8bc-4b63-82b5-5667a7eac83f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:09.178652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e9306df7-d8bc-4b63-82b5-5667a7eac83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:09.239696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:09.352415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9306df7-d8bc-4b63-82b5-5667a7eac83f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:09.496937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cfc70-29a8-4f51-9ee5-7f3e836309bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:09.497753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cfc70-29a8-4f51-9ee5-7f3e836309bb service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:09.498156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cfc70-29a8-4f51-9ee5-7f3e836309bb service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:09.498538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cfc70-29a8-4f51-9ee5-7f3e836309bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:09.500318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e29cfc70-29a8-4f51-9ee5-7f3e836309bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:09.520195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:09.597498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cfc70-29a8-4f51-9ee5-7f3e836309bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:09.599235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cfc70-29a8-4f51-9ee5-7f3e836309bb service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:09.622044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 691/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:11.845710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3e975-88db-4ae5-ab23-3c18dfcb0080 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:11.846418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3e975-88db-4ae5-ab23-3c18dfcb0080 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:11.846418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3e975-88db-4ae5-ab23-3c18dfcb0080 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:11.847898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 689/1380] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:11.853124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531cec-fa81-42a7-9fa3-21cc81650fc6 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:11.853547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531cec-fa81-42a7-9fa3-21cc81650fc6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:11.853730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531cec-fa81-42a7-9fa3-21cc81650fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:11.920828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-46531cec-fa81-42a7-9fa3-21cc81650fc6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:11.922309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-46531cec-fa81-42a7-9fa3-21cc81650fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:12.019605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:12.025837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 692/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:12.071118 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3e975-88db-4ae5-ab23-3c18dfcb0080 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:12.072784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2cd3e975-88db-4ae5-ab23-3c18dfcb0080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:12.225353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:12.239885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3e975-88db-4ae5-ab23-3c18dfcb0080 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:12.369567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8f14e-a8cf-4963-826c-e7d277576810 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:12.370120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f14e-a8cf-4963-826c-e7d277576810 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:12.370404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f14e-a8cf-4963-826c-e7d277576810 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:12.370680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8f14e-a8cf-4963-826c-e7d277576810 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:12.372357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7bf8f14e-a8cf-4963-826c-e7d277576810 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:12.467866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8f14e-a8cf-4963-826c-e7d277576810 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:12.469495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8f14e-a8cf-4963-826c-e7d277576810 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:12.480926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:12.489697 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 690/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:14.700314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8383d-744b-4c80-b6f2-8415a4253e04 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:14.700917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8383d-744b-4c80-b6f2-8415a4253e04 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:14.701032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8383d-744b-4c80-b6f2-8415a4253e04 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:14.702957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 693/1383] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:14.708399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45445699-7933-4899-8d9e-b3f32b0b12cb None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:14.708806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45445699-7933-4899-8d9e-b3f32b0b12cb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:14.708958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45445699-7933-4899-8d9e-b3f32b0b12cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:14.776474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-45445699-7933-4899-8d9e-b3f32b0b12cb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:14.777700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-45445699-7933-4899-8d9e-b3f32b0b12cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:14.894978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 691/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:52:14.949439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8383d-744b-4c80-b6f2-8415a4253e04 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:14.951759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-dbc8383d-744b-4c80-b6f2-8415a4253e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:15.105925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8383d-744b-4c80-b6f2-8415a4253e04 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:15.172971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:15.224750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f003e7d-2491-4b2c-90a9-34e13a947a89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:15.225578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f003e7d-2491-4b2c-90a9-34e13a947a89 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:15.225950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f003e7d-2491-4b2c-90a9-34e13a947a89 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:15.226316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f003e7d-2491-4b2c-90a9-34e13a947a89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:15.228622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4f003e7d-2491-4b2c-90a9-34e13a947a89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:15.334557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f003e7d-2491-4b2c-90a9-34e13a947a89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:15.336210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f003e7d-2491-4b2c-90a9-34e13a947a89 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:15.354357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 694/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:17.415167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f21767-f529-4db5-a962-42ffb9f36bd4 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:17.415769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f21767-f529-4db5-a962-42ffb9f36bd4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:17.415913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f21767-f529-4db5-a962-42ffb9f36bd4 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:17.417938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 692/1386] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:17.423274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b013d68-2920-4e43-a9f0-6609c46970bc None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:17.423699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b013d68-2920-4e43-a9f0-6609c46970bc None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:17.423868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b013d68-2920-4e43-a9f0-6609c46970bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:17.501310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2b013d68-2920-4e43-a9f0-6609c46970bc None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:17.503003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2b013d68-2920-4e43-a9f0-6609c46970bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:17.539658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:17.607110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 695/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:17.653152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f21767-f529-4db5-a962-42ffb9f36bd4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:17.655144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-57f21767-f529-4db5-a962-42ffb9f36bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:17.758843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:17.836423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f21767-f529-4db5-a962-42ffb9f36bd4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:17.963955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc1e72-ccfc-43d4-8bda-5927e6dc7e43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:17.965162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc1e72-ccfc-43d4-8bda-5927e6dc7e43 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:17.965532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc1e72-ccfc-43d4-8bda-5927e6dc7e43 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:17.965917 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc1e72-ccfc-43d4-8bda-5927e6dc7e43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:17.968392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-06fc1e72-ccfc-43d4-8bda-5927e6dc7e43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:18.018535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:18.072016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc1e72-ccfc-43d4-8bda-5927e6dc7e43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:18.073711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc1e72-ccfc-43d4-8bda-5927e6dc7e43 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:18.095239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 693/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:20.316820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957307d6-b004-4aa6-b86e-8624f18d3f4c None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:20.317450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957307d6-b004-4aa6-b86e-8624f18d3f4c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:20.317450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957307d6-b004-4aa6-b86e-8624f18d3f4c None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:20.318946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 696/1389] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:20.323942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accc71c-e164-4722-ab74-479bbb84b9a7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:20.324397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accc71c-e164-4722-ab74-479bbb84b9a7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:20.324529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accc71c-e164-4722-ab74-479bbb84b9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:20.390724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8accc71c-e164-4722-ab74-479bbb84b9a7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:20.391941 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8accc71c-e164-4722-ab74-479bbb84b9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:20.491526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 694/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:20.537902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957307d6-b004-4aa6-b86e-8624f18d3f4c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:20.539517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-957307d6-b004-4aa6-b86e-8624f18d3f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:20.690411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957307d6-b004-4aa6-b86e-8624f18d3f4c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:20.707148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:20.811538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784b7cd-191d-4a7b-a212-856793389149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:20.812243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784b7cd-191d-4a7b-a212-856793389149 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:20.812531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784b7cd-191d-4a7b-a212-856793389149 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:20.812807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784b7cd-191d-4a7b-a212-856793389149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:20.814528 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9784b7cd-191d-4a7b-a212-856793389149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:20.904871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784b7cd-191d-4a7b-a212-856793389149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:20.906480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784b7cd-191d-4a7b-a212-856793389149 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:20.924742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 697/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:21.129726 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabae169-2355-4be1-afe1-940bb663ead9 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:21.130021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabae169-2355-4be1-afe1-940bb663ead9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:21.130303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabae169-2355-4be1-afe1-940bb663ead9 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:21.131676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 695/1392] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:21.137010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7a9bf-b6a9-466e-9c29-1d7757ad14ab None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:21.137302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7a9bf-b6a9-466e-9c29-1d7757ad14ab None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:21.137580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7a9bf-b6a9-466e-9c29-1d7757ad14ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:21.165127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:21.214382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d5c7a9bf-b6a9-466e-9c29-1d7757ad14ab None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:21.215625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d5c7a9bf-b6a9-466e-9c29-1d7757ad14ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:21.319167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 698/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:21.376266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73824bdb-03dd-4f77-bd3a-84dcf93a8488 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:21.376738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73824bdb-03dd-4f77-bd3a-84dcf93a8488 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:21.376842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73824bdb-03dd-4f77-bd3a-84dcf93a8488 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:21.379048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 696/1394] 10.222.0.40 () {54 vars in 966 bytes} [Mon Jul 25 05:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:21.384196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaa874-c75f-48a9-a746-3757d873170c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:21.384487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaa874-c75f-48a9-a746-3757d873170c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:21.384780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdaa874-c75f-48a9-a746-3757d873170c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:21.459622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-7fdaa874-c75f-48a9-a746-3757d873170c None None] MFA Rules not processed for user `3c0883bc26a04de2916c78df342d2023`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:21.460854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7fdaa874-c75f-48a9-a746-3757d873170c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:21.501388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:21.537663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 699/1395] 10.222.0.40 () {58 vars in 1063 bytes} [Mon Jul 25 05:52:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:21.545710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73824bdb-03dd-4f77-bd3a-84dcf93a8488 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:21.547501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-73824bdb-03dd-4f77-bd3a-84dcf93a8488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:21.561877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:21.652205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73824bdb-03dd-4f77-bd3a-84dcf93a8488 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:21.727345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47432d19-2fb1-40ca-9664-597f4eeae6df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c0883bc26a04de2916c78df342d2023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:21.728005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432d19-2fb1-40ca-9664-597f4eeae6df service glance] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:21.728321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432d19-2fb1-40ca-9664-597f4eeae6df service glance] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:21.728600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47432d19-2fb1-40ca-9664-597f4eeae6df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:21.730256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-47432d19-2fb1-40ca-9664-597f4eeae6df service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:21.804022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:21.850355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47432d19-2fb1-40ca-9664-597f4eeae6df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:21.851890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47432d19-2fb1-40ca-9664-597f4eeae6df service glance] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:21.871684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 697/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:22.722282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f2cfc-596a-4c87-b7d5-8283fa8f56f3 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:22.722750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f2cfc-596a-4c87-b7d5-8283fa8f56f3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:22.725920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f2cfc-596a-4c87-b7d5-8283fa8f56f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:22.789665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d95f2cfc-596a-4c87-b7d5-8283fa8f56f3 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:22.790881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d95f2cfc-596a-4c87-b7d5-8283fa8f56f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:22.881828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 700/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 05:52:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:22.929877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc7525-7ce6-4a2f-8f28-d2695a12d28e None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:22.930361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc7525-7ce6-4a2f-8f28-d2695a12d28e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:22.930491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdc7525-7ce6-4a2f-8f28-d2695a12d28e None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:22.931862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 698/1398] 10.222.0.40 () {54 vars in 943 bytes} [Mon Jul 25 05:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:22.938749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ba49-a48c-4516-b272-57e6d65c0dae None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:22.939035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ba49-a48c-4516-b272-57e6d65c0dae None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:22.943672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268ba49-a48c-4516-b272-57e6d65c0dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:23.010723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:23.018479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-e268ba49-a48c-4516-b272-57e6d65c0dae None None] MFA Rules not processed for user `cf71e1ef5acb429b96219888830181d3`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:23.019695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e268ba49-a48c-4516-b272-57e6d65c0dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:23.131562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 701/1399] 10.222.0.40 () {58 vars in 1040 bytes} [Mon Jul 25 05:52:22 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:23.143953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc7525-7ce6-4a2f-8f28-d2695a12d28e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:23.145575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2bdc7525-7ce6-4a2f-8f28-d2695a12d28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:23.159464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4429b-5156-4227-9ebc-20d97a5a0a78 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:23.159752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4429b-5156-4227-9ebc-20d97a5a0a78 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:23.160033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4429b-5156-4227-9ebc-20d97a5a0a78 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:23.161453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 702/1400] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:23.166526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa707-da69-4027-91a0-43e0d582b852 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:23.166816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa707-da69-4027-91a0-43e0d582b852 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:23.167098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa707-da69-4027-91a0-43e0d582b852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:23.235387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-26dfa707-da69-4027-91a0-43e0d582b852 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:23.237399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-26dfa707-da69-4027-91a0-43e0d582b852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:23.308217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdc7525-7ce6-4a2f-8f28-d2695a12d28e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:23.310665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:23.350630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 703/1401] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:52:23.351507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:23.398886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfa707-da69-4027-91a0-43e0d582b852 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:23.400101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-26dfa707-da69-4027-91a0-43e0d582b852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:23.430771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e8731-7b84-4c51-8b72-4f9aafcfaa98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:23.431445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e8731-7b84-4c51-8b72-4f9aafcfaa98 service swift] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:23.431720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e8731-7b84-4c51-8b72-4f9aafcfaa98 service swift] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:23.431987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e8731-7b84-4c51-8b72-4f9aafcfaa98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:23.433743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0b5e8731-7b84-4c51-8b72-4f9aafcfaa98 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:23.523204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e8731-7b84-4c51-8b72-4f9aafcfaa98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:23.524969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e8731-7b84-4c51-8b72-4f9aafcfaa98 service swift] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:23.532475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfa707-da69-4027-91a0-43e0d582b852 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:23.534824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 699/1402] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 05:52:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:52:23.625774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:23.679428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55667a-612c-4fbd-80ef-9ca2a9782eb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:23.684499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55667a-612c-4fbd-80ef-9ca2a9782eb7 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:23.684782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55667a-612c-4fbd-80ef-9ca2a9782eb7 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:23.685107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55667a-612c-4fbd-80ef-9ca2a9782eb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:23.686801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0f55667a-612c-4fbd-80ef-9ca2a9782eb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:23.820661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55667a-612c-4fbd-80ef-9ca2a9782eb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:23.822343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55667a-612c-4fbd-80ef-9ca2a9782eb7 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:23.844530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 704/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:52:27.149374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56f861-4ed6-45cf-a48a-383d590954a2 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:27.149945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56f861-4ed6-45cf-a48a-383d590954a2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:27.150240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56f861-4ed6-45cf-a48a-383d590954a2 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:27.151680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 700/1404] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:27.157171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc551e-11c5-4d59-9b98-632ec7d906e2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:27.157638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc551e-11c5-4d59-9b98-632ec7d906e2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:27.157804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc551e-11c5-4d59-9b98-632ec7d906e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:27.235479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-52cc551e-11c5-4d59-9b98-632ec7d906e2 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:27.238163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-52cc551e-11c5-4d59-9b98-632ec7d906e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:27.290210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:27.364231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 705/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:27.410905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56f861-4ed6-45cf-a48a-383d590954a2 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:27.412901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ef56f861-4ed6-45cf-a48a-383d590954a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:27.497713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:27.590201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56f861-4ed6-45cf-a48a-383d590954a2 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:27.720282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ce61e0-0014-41ae-906c-7e3abf1d6e4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:27.721752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce61e0-0014-41ae-906c-7e3abf1d6e4a service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:27.722099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce61e0-0014-41ae-906c-7e3abf1d6e4a service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:27.722387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce61e0-0014-41ae-906c-7e3abf1d6e4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:27.724769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-68ce61e0-0014-41ae-906c-7e3abf1d6e4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:27.766181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:27.837516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce61e0-0014-41ae-906c-7e3abf1d6e4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:27.839220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ce61e0-0014-41ae-906c-7e3abf1d6e4a service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:27.861787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 701/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:30.259878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93f551-311a-4516-87a0-fd724b2f5908 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:30.259878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93f551-311a-4516-87a0-fd724b2f5908 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:30.259878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da93f551-311a-4516-87a0-fd724b2f5908 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:30.260814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 706/1407] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:30.266167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91816d-7c90-4599-a6ce-19b076d82d21 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:30.266575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91816d-7c90-4599-a6ce-19b076d82d21 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:30.266780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91816d-7c90-4599-a6ce-19b076d82d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:30.338898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-0b91816d-7c90-4599-a6ce-19b076d82d21 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:30.339264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0b91816d-7c90-4599-a6ce-19b076d82d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:30.450723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 702/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:30.496095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93f551-311a-4516-87a0-fd724b2f5908 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:30.497673 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-da93f551-311a-4516-87a0-fd724b2f5908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:30.671802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da93f551-311a-4516-87a0-fd724b2f5908 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:30.696004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:30.808104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85910a20-862f-4684-9c65-766eb33d6cbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:30.809127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910a20-862f-4684-9c65-766eb33d6cbd service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:30.809702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910a20-862f-4684-9c65-766eb33d6cbd service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:30.810418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85910a20-862f-4684-9c65-766eb33d6cbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:30.812980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-85910a20-862f-4684-9c65-766eb33d6cbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:30.915420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85910a20-862f-4684-9c65-766eb33d6cbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:30.917107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85910a20-862f-4684-9c65-766eb33d6cbd service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:30.936680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 707/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:40.448998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:40.933928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=88 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:47.793550 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8398-8443-409b-92e4-a5f4ca9ddeeb None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:47.794096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8398-8443-409b-92e4-a5f4ca9ddeeb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:47.794096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a8398-8443-409b-92e4-a5f4ca9ddeeb None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:47.795743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 703/1410] 10.222.0.40 () {54 vars in 918 bytes} [Mon Jul 25 05:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:47.801695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95c3eb-60f3-4c4d-a3e2-2aacca615563 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:47.802108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95c3eb-60f3-4c4d-a3e2-2aacca615563 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:47.802250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95c3eb-60f3-4c4d-a3e2-2aacca615563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:47.875518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-7c95c3eb-60f3-4c4d-a3e2-2aacca615563 None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:47.876778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7c95c3eb-60f3-4c4d-a3e2-2aacca615563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:47.980813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 708/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Mon Jul 25 05:52:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:47.996004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111eef7a-209b-48c5-aff6-31dd05c33efc None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:47.996325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111eef7a-209b-48c5-aff6-31dd05c33efc None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:47.996639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111eef7a-209b-48c5-aff6-31dd05c33efc None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:47.998031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 704/1412] 10.222.0.40 () {54 vars in 918 bytes} [Mon Jul 25 05:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:48.002662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1257a-13de-4173-a79b-34084956d890 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:48.002949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1257a-13de-4173-a79b-34084956d890 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:48.003228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1257a-13de-4173-a79b-34084956d890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:48.073772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:48.076347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fab1257a-13de-4173-a79b-34084956d890 None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:48.077555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fab1257a-13de-4173-a79b-34084956d890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:48.192665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 709/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Mon Jul 25 05:52:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:50.682436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60323a5-fe20-4384-8cbe-08d84486b0a9 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:50.683006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60323a5-fe20-4384-8cbe-08d84486b0a9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:50.683006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60323a5-fe20-4384-8cbe-08d84486b0a9 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:50.684386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:50.690219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b543-5c18-44d1-b41a-e4b6712f224e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:50.690662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b543-5c18-44d1-b41a-e4b6712f224e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:50.690770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b543-5c18-44d1-b41a-e4b6712f224e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:50.753617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-51d5b543-5c18-44d1-b41a-e4b6712f224e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:50.754828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-51d5b543-5c18-44d1-b41a-e4b6712f224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:50.809969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:50.851157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:50.893618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60323a5-fe20-4384-8cbe-08d84486b0a9 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:50.895181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f60323a5-fe20-4384-8cbe-08d84486b0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:51.042924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60323a5-fe20-4384-8cbe-08d84486b0a9 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:51.116265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:51.156401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c422e9d-667a-4389-ae01-47ae215cb404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:51.157052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c422e9d-667a-4389-ae01-47ae215cb404 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:51.158068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c422e9d-667a-4389-ae01-47ae215cb404 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:51.158431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c422e9d-667a-4389-ae01-47ae215cb404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:51.160211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7c422e9d-667a-4389-ae01-47ae215cb404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:51.249094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c422e9d-667a-4389-ae01-47ae215cb404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:51.250675 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c422e9d-667a-4389-ae01-47ae215cb404 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:51.268865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 706/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:52.169171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934e27-7aeb-4154-8aa2-543e1f83462f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:52.169723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934e27-7aeb-4154-8aa2-543e1f83462f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:52.169723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3934e27-7aeb-4154-8aa2-543e1f83462f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:52.171369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:52.176586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1474b-e563-4733-9736-398374161c6f None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:52.177034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1474b-e563-4733-9736-398374161c6f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:52.177146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1474b-e563-4733-9736-398374161c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:52.250310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-11c1474b-e563-4733-9736-398374161c6f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:52.252034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-11c1474b-e563-4733-9736-398374161c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:52.278399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:52.347239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:52.494812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3934e27-7aeb-4154-8aa2-543e1f83462f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:52.496294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e3934e27-7aeb-4154-8aa2-543e1f83462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:52.639506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3934e27-7aeb-4154-8aa2-543e1f83462f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:52.678929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:52.752276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4937b0-51c4-46e5-99c5-426d734fae50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:52.752872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4937b0-51c4-46e5-99c5-426d734fae50 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:52.753160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4937b0-51c4-46e5-99c5-426d734fae50 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:52.753440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4937b0-51c4-46e5-99c5-426d734fae50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:52.755213 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9b4937b0-51c4-46e5-99c5-426d734fae50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:52.846793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4937b0-51c4-46e5-99c5-426d734fae50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:52.848488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4937b0-51c4-46e5-99c5-426d734fae50 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:52.866997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 712/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:53.757824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49276ad6-0d99-46cd-9aed-09629223a0c6 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:53.758388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49276ad6-0d99-46cd-9aed-09629223a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:53.758388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49276ad6-0d99-46cd-9aed-09629223a0c6 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:53.759747 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:52:53.764750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4627a-779c-4b26-843d-80369704378e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:53.765151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4627a-779c-4b26-843d-80369704378e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:53.765421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4627a-779c-4b26-843d-80369704378e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:53.819369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:53.840679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c8c4627a-779c-4b26-843d-80369704378e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:53.841880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c8c4627a-779c-4b26-843d-80369704378e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:53.937547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:54.088550 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49276ad6-0d99-46cd-9aed-09629223a0c6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:54.090027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-49276ad6-0d99-46cd-9aed-09629223a0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:54.233716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49276ad6-0d99-46cd-9aed-09629223a0c6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:54.245828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:54.345654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff189e-fd87-430f-bbb2-024b5de4c45a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:54.346228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff189e-fd87-430f-bbb2-024b5de4c45a service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:54.346509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff189e-fd87-430f-bbb2-024b5de4c45a service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:54.346783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff189e-fd87-430f-bbb2-024b5de4c45a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:54.348466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d8ff189e-fd87-430f-bbb2-024b5de4c45a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:54.435736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff189e-fd87-430f-bbb2-024b5de4c45a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:54.437348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff189e-fd87-430f-bbb2-024b5de4c45a service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:54.455595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 709/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:55.320471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f544c-9663-4f1c-abe2-86ae0c3bba59 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:55.321047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f544c-9663-4f1c-abe2-86ae0c3bba59 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:55.321047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f544c-9663-4f1c-abe2-86ae0c3bba59 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:55.322411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:55.327627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83f432-6dbe-4dbf-9ab1-00ffaddb3ef8 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:55.328059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83f432-6dbe-4dbf-9ab1-00ffaddb3ef8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:55.328218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83f432-6dbe-4dbf-9ab1-00ffaddb3ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:55.347264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:55.402123 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3f83f432-6dbe-4dbf-9ab1-00ffaddb3ef8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:55.403335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3f83f432-6dbe-4dbf-9ab1-00ffaddb3ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:55.499671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:55.649014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f544c-9663-4f1c-abe2-86ae0c3bba59 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:55.650518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-743f544c-9663-4f1c-abe2-86ae0c3bba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:55.760177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:55.798088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f544c-9663-4f1c-abe2-86ae0c3bba59 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:55.911570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1baf3-8621-4e5f-b332-3ebdb6da543c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:55.912259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1baf3-8621-4e5f-b332-3ebdb6da543c service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:55.912551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1baf3-8621-4e5f-b332-3ebdb6da543c service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:55.912834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1baf3-8621-4e5f-b332-3ebdb6da543c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:55.914501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f6f1baf3-8621-4e5f-b332-3ebdb6da543c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:55.993038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:56.006881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1baf3-8621-4e5f-b332-3ebdb6da543c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:56.008520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1baf3-8621-4e5f-b332-3ebdb6da543c service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:56.027378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 715/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:56.888255 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616b86-1f30-475b-a300-922190c58fd4 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:56.888841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616b86-1f30-475b-a300-922190c58fd4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:56.888841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8616b86-1f30-475b-a300-922190c58fd4 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:56.890190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:56.895054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2d9f-569d-445b-8677-8f08a1ecd2ab None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:56.895555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2d9f-569d-445b-8677-8f08a1ecd2ab None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:56.895663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2d9f-569d-445b-8677-8f08a1ecd2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:56.968499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ecdf2d9f-569d-445b-8677-8f08a1ecd2ab None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:56.970239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ecdf2d9f-569d-445b-8677-8f08a1ecd2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:57.068660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:57.218175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8616b86-1f30-475b-a300-922190c58fd4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:57.220376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a8616b86-1f30-475b-a300-922190c58fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:57.304798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:57.365608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8616b86-1f30-475b-a300-922190c58fd4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:57.476581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081c6fc-e158-4a13-9069-37660c08a42a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:57.477233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c6fc-e158-4a13-9069-37660c08a42a service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:57.477516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c6fc-e158-4a13-9069-37660c08a42a service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:57.477792 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081c6fc-e158-4a13-9069-37660c08a42a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:57.479451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9081c6fc-e158-4a13-9069-37660c08a42a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:57.527525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:57.568292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081c6fc-e158-4a13-9069-37660c08a42a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:57.569823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081c6fc-e158-4a13-9069-37660c08a42a service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:57.588516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 712/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:52:58.454691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532fd5a-ec7f-4528-b715-8db2f027a043 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:58.455280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532fd5a-ec7f-4528-b715-8db2f027a043 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:58.455280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532fd5a-ec7f-4528-b715-8db2f027a043 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:58.456678 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:52:58.461676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6403b2-643f-4a28-a631-7eba9de84ac1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:58.462077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6403b2-643f-4a28-a631-7eba9de84ac1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:58.462234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6403b2-643f-4a28-a631-7eba9de84ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:58.538410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1c6403b2-643f-4a28-a631-7eba9de84ac1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:52:58.540404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1c6403b2-643f-4a28-a631-7eba9de84ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:58.637742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:52:58.790389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532fd5a-ec7f-4528-b715-8db2f027a043 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:52:58.791908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e532fd5a-ec7f-4528-b715-8db2f027a043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:58.839772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:58.935936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532fd5a-ec7f-4528-b715-8db2f027a043 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:52:59.046898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e96af3-c278-49f3-b85f-41c542adab22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:52:59.047534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e96af3-c278-49f3-b85f-41c542adab22 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:52:59.047788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e96af3-c278-49f3-b85f-41c542adab22 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:52:59.048067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e96af3-c278-49f3-b85f-41c542adab22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:52:59.049767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-22e96af3-c278-49f3-b85f-41c542adab22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:52:59.056592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:52:59.140076 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e96af3-c278-49f3-b85f-41c542adab22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:52:59.142658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e96af3-c278-49f3-b85f-41c542adab22 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:52:59.160505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 718/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:00.050915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4583c-92d6-4b98-beb9-3f960035f2e3 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:00.051465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4583c-92d6-4b98-beb9-3f960035f2e3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:00.051465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4583c-92d6-4b98-beb9-3f960035f2e3 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:00.052785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:00.057805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97fbf2-b508-46c6-a5f3-6596cfbd1f8c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:00.058221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97fbf2-b508-46c6-a5f3-6596cfbd1f8c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:00.058372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97fbf2-b508-46c6-a5f3-6596cfbd1f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:00.131246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-cf97fbf2-b508-46c6-a5f3-6596cfbd1f8c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:00.132497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cf97fbf2-b508-46c6-a5f3-6596cfbd1f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:00.199197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:00.228418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:00.380918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4583c-92d6-4b98-beb9-3f960035f2e3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:00.382353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c6a4583c-92d6-4b98-beb9-3f960035f2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:00.400783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:00.530014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4583c-92d6-4b98-beb9-3f960035f2e3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:00.616184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:00.642315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23650650-5dd5-49bc-bb55-c8ac7067f945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:00.642904 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650650-5dd5-49bc-bb55-c8ac7067f945 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:00.643189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650650-5dd5-49bc-bb55-c8ac7067f945 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:00.643472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23650650-5dd5-49bc-bb55-c8ac7067f945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:00.645150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-23650650-5dd5-49bc-bb55-c8ac7067f945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:00.733091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23650650-5dd5-49bc-bb55-c8ac7067f945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:00.734685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23650650-5dd5-49bc-bb55-c8ac7067f945 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:00.752984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 715/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:01.596280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2455-ffd2-451d-bd5c-9435945d6db6 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:01.596849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2455-ffd2-451d-bd5c-9435945d6db6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:01.596849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c2455-ffd2-451d-bd5c-9435945d6db6 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:01.598229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:01.603183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb340fc-5716-4943-b169-40d839b699fd None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:01.603612 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb340fc-5716-4943-b169-40d839b699fd None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:01.603725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb340fc-5716-4943-b169-40d839b699fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:01.678049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-fcb340fc-5716-4943-b169-40d839b699fd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:01.679266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fcb340fc-5716-4943-b169-40d839b699fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:01.720031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:01.777857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:53:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:01.928789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c2455-ffd2-451d-bd5c-9435945d6db6 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:01.930379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-276c2455-ffd2-451d-bd5c-9435945d6db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:02.074630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c2455-ffd2-451d-bd5c-9435945d6db6 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:02.127307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:02.186994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7470-d528-4484-9600-f207563923c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:02.187566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7470-d528-4484-9600-f207563923c1 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:02.187893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7470-d528-4484-9600-f207563923c1 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:02.188351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7470-d528-4484-9600-f207563923c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:02.190021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-77db7470-d528-4484-9600-f207563923c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:02.279927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db7470-d528-4484-9600-f207563923c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:02.282115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db7470-d528-4484-9600-f207563923c1 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:02.301083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 721/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:03.178674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef7ec5-677b-4dac-9144-197b33a67ae3 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:03.179251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef7ec5-677b-4dac-9144-197b33a67ae3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:03.179251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef7ec5-677b-4dac-9144-197b33a67ae3 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:03.180638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:03.185496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9b990-555b-44f0-8adf-1fd63a22d4bd None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:03.185948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9b990-555b-44f0-8adf-1fd63a22d4bd None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:03.186079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9b990-555b-44f0-8adf-1fd63a22d4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:03.259153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b3c9b990-555b-44f0-8adf-1fd63a22d4bd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:03.260397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b3c9b990-555b-44f0-8adf-1fd63a22d4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:03.262376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:03.358681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:03.509077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef7ec5-677b-4dac-9144-197b33a67ae3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:03.510610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9aef7ec5-677b-4dac-9144-197b33a67ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:03.659199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef7ec5-677b-4dac-9144-197b33a67ae3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:03.685816 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:03.772135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d0d4c-fbec-4d25-8c56-baab94214694 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:03.772805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d0d4c-fbec-4d25-8c56-baab94214694 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:03.773090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d0d4c-fbec-4d25-8c56-baab94214694 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:03.773363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d0d4c-fbec-4d25-8c56-baab94214694 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:03.775097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2a3d0d4c-fbec-4d25-8c56-baab94214694 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:03.863937 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d0d4c-fbec-4d25-8c56-baab94214694 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:03.865561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d0d4c-fbec-4d25-8c56-baab94214694 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:03.884215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 718/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:05.829101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b137-bc8b-4086-8a90-04b9037f07c5 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:05.829688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b137-bc8b-4086-8a90-04b9037f07c5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:05.829688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2b137-bc8b-4086-8a90-04b9037f07c5 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:05.831197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:05.836347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258176b-72ba-4c11-af10-9edf016ff3f5 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:05.836736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258176b-72ba-4c11-af10-9edf016ff3f5 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:05.836915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258176b-72ba-4c11-af10-9edf016ff3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:05.877488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:05.901841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f258176b-72ba-4c11-af10-9edf016ff3f5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:05.903046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f258176b-72ba-4c11-af10-9edf016ff3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:06.005655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:06.044837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2b137-bc8b-4086-8a90-04b9037f07c5 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:06.046446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3ba2b137-bc8b-4086-8a90-04b9037f07c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:06.150596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2b137-bc8b-4086-8a90-04b9037f07c5 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:06.183386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:06.225847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a4fba-5f8e-4f38-ab96-2af2c114ca12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:06.226522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a4fba-5f8e-4f38-ab96-2af2c114ca12 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:06.226791 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a4fba-5f8e-4f38-ab96-2af2c114ca12 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:06.227070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a4fba-5f8e-4f38-ab96-2af2c114ca12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:06.228768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-419a4fba-5f8e-4f38-ab96-2af2c114ca12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:06.337700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a4fba-5f8e-4f38-ab96-2af2c114ca12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:06.339066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a4fba-5f8e-4f38-ab96-2af2c114ca12 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:06.358057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 724/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:08.539993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80faa3-9096-42db-9bc0-d60a5915795d None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:08.540588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80faa3-9096-42db-9bc0-d60a5915795d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:08.540700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80faa3-9096-42db-9bc0-d60a5915795d None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:08.542015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:08.547611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b337a4d-97f7-496d-9e5a-6a2755548203 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:08.548055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b337a4d-97f7-496d-9e5a-6a2755548203 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:08.548210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b337a4d-97f7-496d-9e5a-6a2755548203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:08.612513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4b337a4d-97f7-496d-9e5a-6a2755548203 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:08.613745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4b337a4d-97f7-496d-9e5a-6a2755548203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:08.626374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:08.710282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:08.749036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80faa3-9096-42db-9bc0-d60a5915795d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:08.750620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1a80faa3-9096-42db-9bc0-d60a5915795d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:08.853171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a80faa3-9096-42db-9bc0-d60a5915795d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:08.865445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:08.925483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311d583-8fd1-4e9f-b783-4bd5a8777d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:08.926119 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311d583-8fd1-4e9f-b783-4bd5a8777d75 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:08.926399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311d583-8fd1-4e9f-b783-4bd5a8777d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:08.926671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311d583-8fd1-4e9f-b783-4bd5a8777d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:08.928345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5311d583-8fd1-4e9f-b783-4bd5a8777d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:09.034109 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311d583-8fd1-4e9f-b783-4bd5a8777d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:09.035476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311d583-8fd1-4e9f-b783-4bd5a8777d75 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:09.054120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 721/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:11.416450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba4b0-827d-4950-82cf-754440f7bf2f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:11.417013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba4b0-827d-4950-82cf-754440f7bf2f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:11.417013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adba4b0-827d-4950-82cf-754440f7bf2f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:11.418379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:11.425320 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae968a0-4dc8-4fcb-a104-110766a90846 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:11.425754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae968a0-4dc8-4fcb-a104-110766a90846 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:11.425886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae968a0-4dc8-4fcb-a104-110766a90846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:11.472395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:11.489795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2ae968a0-4dc8-4fcb-a104-110766a90846 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:11.492225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2ae968a0-4dc8-4fcb-a104-110766a90846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:11.585750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:11.624455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adba4b0-827d-4950-82cf-754440f7bf2f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:11.626331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6adba4b0-827d-4950-82cf-754440f7bf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:11.730035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adba4b0-827d-4950-82cf-754440f7bf2f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:11.773112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:11.808136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ed0c1-d195-4f17-9153-35582aa39070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:11.808813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ed0c1-d195-4f17-9153-35582aa39070 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:11.809097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ed0c1-d195-4f17-9153-35582aa39070 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:11.809376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ed0c1-d195-4f17-9153-35582aa39070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:11.811017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9b0ed0c1-d195-4f17-9153-35582aa39070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:11.917749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ed0c1-d195-4f17-9153-35582aa39070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:11.919466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ed0c1-d195-4f17-9153-35582aa39070 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:11.939619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 727/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:14.092541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ab26d-187b-4006-b1dc-806ffa014cca None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:14.093064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ab26d-187b-4006-b1dc-806ffa014cca None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:14.093064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597ab26d-187b-4006-b1dc-806ffa014cca None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:14.094398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:14.100078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df63125-2476-47fc-b09c-6def970d9e5f None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:14.100526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df63125-2476-47fc-b09c-6def970d9e5f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:14.100681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df63125-2476-47fc-b09c-6def970d9e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:14.163971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2df63125-2476-47fc-b09c-6def970d9e5f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:14.165206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2df63125-2476-47fc-b09c-6def970d9e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:14.185774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:14.262161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:14.301864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ab26d-187b-4006-b1dc-806ffa014cca None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:14.303407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-597ab26d-187b-4006-b1dc-806ffa014cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:14.405893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597ab26d-187b-4006-b1dc-806ffa014cca None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:14.426353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:14.480847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5d7ca-95f6-4dc3-81f8-288af0ccf754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:14.481500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5d7ca-95f6-4dc3-81f8-288af0ccf754 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:14.481775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5d7ca-95f6-4dc3-81f8-288af0ccf754 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:14.482056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5d7ca-95f6-4dc3-81f8-288af0ccf754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:14.483719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-52b5d7ca-95f6-4dc3-81f8-288af0ccf754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:14.592727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5d7ca-95f6-4dc3-81f8-288af0ccf754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:14.594350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5d7ca-95f6-4dc3-81f8-288af0ccf754 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:14.612630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 724/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:16.712210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3368c0-b2ee-4837-aefc-3e5876591a07 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:16.712784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3368c0-b2ee-4837-aefc-3e5876591a07 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:16.712784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3368c0-b2ee-4837-aefc-3e5876591a07 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:16.714138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:16.720182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d50a2d-6a92-43af-9344-80c1d18e3778 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:16.720574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d50a2d-6a92-43af-9344-80c1d18e3778 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:16.720744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d50a2d-6a92-43af-9344-80c1d18e3778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:16.782759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:16.785686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f2d50a2d-6a92-43af-9344-80c1d18e3778 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:16.786905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f2d50a2d-6a92-43af-9344-80c1d18e3778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:16.883587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:16.922944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3368c0-b2ee-4837-aefc-3e5876591a07 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:16.924528 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3f3368c0-b2ee-4837-aefc-3e5876591a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:17.027257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3368c0-b2ee-4837-aefc-3e5876591a07 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:17.077396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:17.101910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b9eb9-39ba-4fe2-8c00-f838d8374a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:17.102559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b9eb9-39ba-4fe2-8c00-f838d8374a8e service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:17.102849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b9eb9-39ba-4fe2-8c00-f838d8374a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:17.103127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b9eb9-39ba-4fe2-8c00-f838d8374a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:17.104810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8b6b9eb9-39ba-4fe2-8c00-f838d8374a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:17.213165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b9eb9-39ba-4fe2-8c00-f838d8374a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:17.214622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b9eb9-39ba-4fe2-8c00-f838d8374a8e service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:17.233344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 730/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:19.317583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd878ca-c7aa-4349-b34c-9cc34e41e516 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:19.318160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd878ca-c7aa-4349-b34c-9cc34e41e516 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:19.318160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd878ca-c7aa-4349-b34c-9cc34e41e516 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:19.319537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:19.324889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7110ee-16ed-4aa8-922b-d99b4f080ea0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:19.325316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7110ee-16ed-4aa8-922b-d99b4f080ea0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:19.325446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7110ee-16ed-4aa8-922b-d99b4f080ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:19.393572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ca7110ee-16ed-4aa8-922b-d99b4f080ea0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:19.394770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ca7110ee-16ed-4aa8-922b-d99b4f080ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:19.423983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:19.492338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:19.531300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd878ca-c7aa-4349-b34c-9cc34e41e516 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:19.532868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-dbd878ca-c7aa-4349-b34c-9cc34e41e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:19.638457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd878ca-c7aa-4349-b34c-9cc34e41e516 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:19.666689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:19.713695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a310b321-eb0e-446b-afdf-5cef10de0027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:19.714337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b321-eb0e-446b-afdf-5cef10de0027 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:19.714622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b321-eb0e-446b-afdf-5cef10de0027 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:19.714896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a310b321-eb0e-446b-afdf-5cef10de0027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:19.716567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a310b321-eb0e-446b-afdf-5cef10de0027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:19.825314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310b321-eb0e-446b-afdf-5cef10de0027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:19.826756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a310b321-eb0e-446b-afdf-5cef10de0027 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:19.845772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 727/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:21.954026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17328349-7549-4f09-b809-41824e5dfc3a None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:21.954587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17328349-7549-4f09-b809-41824e5dfc3a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:21.954587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17328349-7549-4f09-b809-41824e5dfc3a None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:21.955938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:21.961286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0c06d-847e-4363-93de-d50ba3806336 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:21.961706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0c06d-847e-4363-93de-d50ba3806336 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:21.961912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0c06d-847e-4363-93de-d50ba3806336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:22.026251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-5da0c06d-847e-4363-93de-d50ba3806336 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:22.027370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5da0c06d-847e-4363-93de-d50ba3806336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:22.035601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:22.123447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:22.161305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17328349-7549-4f09-b809-41824e5dfc3a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:22.162842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-17328349-7549-4f09-b809-41824e5dfc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:22.266100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17328349-7549-4f09-b809-41824e5dfc3a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:22.321059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:22.338936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7818176-255d-426e-8eca-2199de619c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:22.339586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7818176-255d-426e-8eca-2199de619c40 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:22.339867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7818176-255d-426e-8eca-2199de619c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:22.340205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7818176-255d-426e-8eca-2199de619c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:22.341867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a7818176-255d-426e-8eca-2199de619c40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:22.451190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7818176-255d-426e-8eca-2199de619c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:22.453354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7818176-255d-426e-8eca-2199de619c40 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:22.471668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 733/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:24.665688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9febc6-8727-451b-8e38-afc81bf42cc9 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:24.666264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9febc6-8727-451b-8e38-afc81bf42cc9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:24.666264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9febc6-8727-451b-8e38-afc81bf42cc9 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:24.667646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:24.673893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1d1b9-230d-4e47-a1cd-6e7e9d6260a7 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:24.674323 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1d1b9-230d-4e47-a1cd-6e7e9d6260a7 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:24.674439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1d1b9-230d-4e47-a1cd-6e7e9d6260a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:24.739496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f7f1d1b9-230d-4e47-a1cd-6e7e9d6260a7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:24.740731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f7f1d1b9-230d-4e47-a1cd-6e7e9d6260a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:24.897441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:24.957125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:53:24.994496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9febc6-8727-451b-8e38-afc81bf42cc9 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:24.996058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bd9febc6-8727-451b-8e38-afc81bf42cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:25.096481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9febc6-8727-451b-8e38-afc81bf42cc9 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:25.132394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:25.167908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f175391-5bbd-47a3-97c7-d7547a3d4397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:25.168588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175391-5bbd-47a3-97c7-d7547a3d4397 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:25.169120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175391-5bbd-47a3-97c7-d7547a3d4397 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:25.169395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f175391-5bbd-47a3-97c7-d7547a3d4397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:25.171020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2f175391-5bbd-47a3-97c7-d7547a3d4397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:25.277407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f175391-5bbd-47a3-97c7-d7547a3d4397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:25.278756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f175391-5bbd-47a3-97c7-d7547a3d4397 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:25.297267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 730/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:27.357307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4203069-a375-48f0-9129-d77b04da702b None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:27.357842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4203069-a375-48f0-9129-d77b04da702b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:27.357842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4203069-a375-48f0-9129-d77b04da702b None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:27.359965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:27.365364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1822a6-4db4-425d-94d5-f570ce03fccf None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:27.365808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1822a6-4db4-425d-94d5-f570ce03fccf None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:27.365953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1822a6-4db4-425d-94d5-f570ce03fccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:27.430400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3f1822a6-4db4-425d-94d5-f570ce03fccf None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:27.431617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3f1822a6-4db4-425d-94d5-f570ce03fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:27.449097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:27.526433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:27.565387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4203069-a375-48f0-9129-d77b04da702b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:27.566921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f4203069-a375-48f0-9129-d77b04da702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:27.669637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4203069-a375-48f0-9129-d77b04da702b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:27.735386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:27.741916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63b9c1-f1f6-4c87-bbb9-aa4fce1c1d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:27.748253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b9c1-f1f6-4c87-bbb9-aa4fce1c1d8e service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:27.748253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b9c1-f1f6-4c87-bbb9-aa4fce1c1d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:27.748253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63b9c1-f1f6-4c87-bbb9-aa4fce1c1d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:27.748253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9b63b9c1-f1f6-4c87-bbb9-aa4fce1c1d8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:27.852078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63b9c1-f1f6-4c87-bbb9-aa4fce1c1d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:27.853552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63b9c1-f1f6-4c87-bbb9-aa4fce1c1d8e service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:27.871605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 736/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:29.924609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e99d6-952b-404a-bcd7-5886ad752116 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:29.925121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e99d6-952b-404a-bcd7-5886ad752116 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:29.925217 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968e99d6-952b-404a-bcd7-5886ad752116 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:29.926542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:29.931998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1f14d-e078-44fc-a053-ba37b821aa7d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:29.932486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1f14d-e078-44fc-a053-ba37b821aa7d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:29.932582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1f14d-e078-44fc-a053-ba37b821aa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:29.997136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-96f1f14d-e078-44fc-a053-ba37b821aa7d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:29.998333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-96f1f14d-e078-44fc-a053-ba37b821aa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:30.048299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:30.096060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:30.134471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e99d6-952b-404a-bcd7-5886ad752116 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:30.136005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-968e99d6-952b-404a-bcd7-5886ad752116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:30.237923 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968e99d6-952b-404a-bcd7-5886ad752116 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:30.285490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:30.310896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380e75d-da12-4570-84b7-30d15f28bf4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:30.311562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e75d-da12-4570-84b7-30d15f28bf4e service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:30.311852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e75d-da12-4570-84b7-30d15f28bf4e service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:30.312153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e75d-da12-4570-84b7-30d15f28bf4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:30.313823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7380e75d-da12-4570-84b7-30d15f28bf4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:30.421158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380e75d-da12-4570-84b7-30d15f28bf4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:30.423445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380e75d-da12-4570-84b7-30d15f28bf4e service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:30.442551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 733/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:40.094311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=22 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:40.440185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=66 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:50.153278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a090c3-295a-42d3-bf73-4ca747f5b37b None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:50.153811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a090c3-295a-42d3-bf73-4ca747f5b37b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:50.153811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a090c3-295a-42d3-bf73-4ca747f5b37b None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:50.155258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 738/1471] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:50.160663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b78191-17b2-4163-99ad-3fc07eb153f9 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:50.161058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b78191-17b2-4163-99ad-3fc07eb153f9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:50.161222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b78191-17b2-4163-99ad-3fc07eb153f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:50.225206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a7b78191-17b2-4163-99ad-3fc07eb153f9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:50.226426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a7b78191-17b2-4163-99ad-3fc07eb153f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:50.321903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 734/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:50.404658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a840-39f6-4f1c-a708-ae30899dd7c5 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:50.405066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a840-39f6-4f1c-a708-ae30899dd7c5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:50.405346 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a840-39f6-4f1c-a708-ae30899dd7c5 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:50.406709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 739/1473] 10.222.0.40 () {54 vars in 966 bytes} [Mon Jul 25 05:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:50.411736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0b46-f986-46dd-ab5e-95853901be24 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:50.412038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0b46-f986-46dd-ab5e-95853901be24 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:50.412354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0b46-f986-46dd-ab5e-95853901be24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:50.488001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d22a0b46-f986-46dd-ab5e-95853901be24 None None] MFA Rules not processed for user `3c0883bc26a04de2916c78df342d2023`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:50.492219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d22a0b46-f986-46dd-ab5e-95853901be24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:50.510233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:50.567085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 735/1474] 10.222.0.40 () {58 vars in 1063 bytes} [Mon Jul 25 05:53:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:50.574567 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3a840-39f6-4f1c-a708-ae30899dd7c5 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:50.576817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a4b3a840-39f6-4f1c-a708-ae30899dd7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:50.679287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3a840-39f6-4f1c-a708-ae30899dd7c5 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:50.750600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f2fa8-4648-4132-af02-193773f95bc8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c0883bc26a04de2916c78df342d2023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:50.751370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f2fa8-4648-4132-af02-193773f95bc8 service glance] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:50.751661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f2fa8-4648-4132-af02-193773f95bc8 service glance] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:50.751939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f2fa8-4648-4132-af02-193773f95bc8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:50.753650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-649f2fa8-4648-4132-af02-193773f95bc8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:50.807891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:50.862221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f2fa8-4648-4132-af02-193773f95bc8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:50.863654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649f2fa8-4648-4132-af02-193773f95bc8 service glance] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:50.881695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 740/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:53.025987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce52c9-b944-4a89-9457-3c2608645511 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:53.026518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce52c9-b944-4a89-9457-3c2608645511 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:53.026518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce52c9-b944-4a89-9457-3c2608645511 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:53.027843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 736/1476] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:53.033819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf4c2a-c4c5-4001-a291-644b84e57442 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:53.034231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf4c2a-c4c5-4001-a291-644b84e57442 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:53.034378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf4c2a-c4c5-4001-a291-644b84e57442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:53.097839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4ddf4c2a-c4c5-4001-a291-644b84e57442 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:53.099048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4ddf4c2a-c4c5-4001-a291-644b84e57442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:53.196722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 741/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:53.220737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce52c9-b944-4a89-9457-3c2608645511 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:53.222298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-08ce52c9-b944-4a89-9457-3c2608645511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:53.346553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce52c9-b944-4a89-9457-3c2608645511 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:53.395525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:53.439809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2406b-a505-4d52-9f90-b43397133bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:53.440493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2406b-a505-4d52-9f90-b43397133bfe admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:53.440805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2406b-a505-4d52-9f90-b43397133bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:53.441090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2406b-a505-4d52-9f90-b43397133bfe admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:53.442098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2406b-a505-4d52-9f90-b43397133bfe admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:53.444395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING py.warnings [None req-dbf2406b-a505-4d52-9f90-b43397133bfe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:53:53.444395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: warnings.warn(msg) Jul 25 05:53:53.444395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: Jul 25 05:53:53.445525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2406b-a505-4d52-9f90-b43397133bfe admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:53.464774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 737/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Mon Jul 25 05:53:53 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:53:55.563405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45eb34-be50-487e-b087-883d40dee551 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:55.563984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45eb34-be50-487e-b087-883d40dee551 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:55.563984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba45eb34-be50-487e-b087-883d40dee551 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:55.565361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 742/1479] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:53:55.570539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6367e-6b30-4a4f-92d6-8d636d21b975 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:55.570946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6367e-6b30-4a4f-92d6-8d636d21b975 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:55.571099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6367e-6b30-4a4f-92d6-8d636d21b975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:55.635455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c6b6367e-6b30-4a4f-92d6-8d636d21b975 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:55.636684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c6b6367e-6b30-4a4f-92d6-8d636d21b975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:55.733422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 738/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:55.776854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45eb34-be50-487e-b087-883d40dee551 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:55.778412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ba45eb34-be50-487e-b087-883d40dee551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:55.788874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:55.930837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba45eb34-be50-487e-b087-883d40dee551 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:56.009667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:56.042798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419da145-0397-4310-87e7-020c9f360ba8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:56.043450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da145-0397-4310-87e7-020c9f360ba8 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:56.043733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da145-0397-4310-87e7-020c9f360ba8 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:56.044008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419da145-0397-4310-87e7-020c9f360ba8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:56.045738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-419da145-0397-4310-87e7-020c9f360ba8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:56.133929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da145-0397-4310-87e7-020c9f360ba8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:56.135437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419da145-0397-4310-87e7-020c9f360ba8 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:56.153605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 743/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:53:58.366052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9d4e-1875-49a6-baee-5e8ac96818bb None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:58.366608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9d4e-1875-49a6-baee-5e8ac96818bb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:58.366608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9d4e-1875-49a6-baee-5e8ac96818bb None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:58.367986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 739/1482] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:53:58.373183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf123-d7ed-414c-80cc-6e3f16961344 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:58.373606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf123-d7ed-414c-80cc-6e3f16961344 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:58.373723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cf123-d7ed-414c-80cc-6e3f16961344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:58.438124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-db5cf123-d7ed-414c-80cc-6e3f16961344 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:53:58.439357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-db5cf123-d7ed-414c-80cc-6e3f16961344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:58.486408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:58.537524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 744/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:53:58.574772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9d4e-1875-49a6-baee-5e8ac96818bb None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:53:58.577428 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-940c9d4e-1875-49a6-baee-5e8ac96818bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:58.601863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:58.679852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9d4e-1875-49a6-baee-5e8ac96818bb None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:53:58.754889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a33f2-e08e-4615-a990-ce4d1d1c3fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:53:58.755568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a33f2-e08e-4615-a990-ce4d1d1c3fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:53:58.755849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a33f2-e08e-4615-a990-ce4d1d1c3fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:53:58.756143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a33f2-e08e-4615-a990-ce4d1d1c3fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:53:58.757822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-af5a33f2-e08e-4615-a990-ce4d1d1c3fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:53:58.831548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:53:58.865427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a33f2-e08e-4615-a990-ce4d1d1c3fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:53:58.866862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a33f2-e08e-4615-a990-ce4d1d1c3fc3 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:53:58.885577 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 740/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:54:00.981131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cddc16-f7c3-4544-82ca-2a8f0655c9d3 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:00.981709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cddc16-f7c3-4544-82ca-2a8f0655c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:00.981709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cddc16-f7c3-4544-82ca-2a8f0655c9d3 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:00.983608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 745/1485] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:54:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:54:00.988876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83b162-7170-4a85-a51b-7c9f00c8e66f None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:00.989290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83b162-7170-4a85-a51b-7c9f00c8e66f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:00.989445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f83b162-7170-4a85-a51b-7c9f00c8e66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:01.053109 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-0f83b162-7170-4a85-a51b-7c9f00c8e66f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:54:01.054324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0f83b162-7170-4a85-a51b-7c9f00c8e66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:01.148659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 741/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:54:01.188227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cddc16-f7c3-4544-82ca-2a8f0655c9d3 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:01.189789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-95cddc16-f7c3-4544-82ca-2a8f0655c9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:01.293368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cddc16-f7c3-4544-82ca-2a8f0655c9d3 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:01.363445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:01.365231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194d83e-37df-4301-8add-c03db60348f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:01.365885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194d83e-37df-4301-8add-c03db60348f4 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:01.366164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194d83e-37df-4301-8add-c03db60348f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:01.366437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194d83e-37df-4301-8add-c03db60348f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:01.368079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8194d83e-37df-4301-8add-c03db60348f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:01.474692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194d83e-37df-4301-8add-c03db60348f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:01.476043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194d83e-37df-4301-8add-c03db60348f4 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:01.494753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 746/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:54:03.687728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47565744-3029-4d9c-ab3b-23faa0e5e611 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:03.688344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47565744-3029-4d9c-ab3b-23faa0e5e611 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:03.688344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47565744-3029-4d9c-ab3b-23faa0e5e611 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:03.689674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 742/1488] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:54:03.694887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716d2f8-b495-4a0a-a959-524416de2017 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:03.695336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716d2f8-b495-4a0a-a959-524416de2017 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:03.695444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716d2f8-b495-4a0a-a959-524416de2017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:03.760220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-5716d2f8-b495-4a0a-a959-524416de2017 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:54:03.761452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5716d2f8-b495-4a0a-a959-524416de2017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:03.813584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:03.858060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 747/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:54:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:54:03.898921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47565744-3029-4d9c-ab3b-23faa0e5e611 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:03.900589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-47565744-3029-4d9c-ab3b-23faa0e5e611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:03.936728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:04.003222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47565744-3029-4d9c-ab3b-23faa0e5e611 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:04.075505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a7929-95c8-4e3a-9729-78de52d5e14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:04.078339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7929-95c8-4e3a-9729-78de52d5e14a service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:04.078339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7929-95c8-4e3a-9729-78de52d5e14a service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:04.078339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7929-95c8-4e3a-9729-78de52d5e14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:04.079193 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3a4a7929-95c8-4e3a-9729-78de52d5e14a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:04.160904 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:04.190020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a7929-95c8-4e3a-9729-78de52d5e14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:04.191448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a7929-95c8-4e3a-9729-78de52d5e14a service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:04.210480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 743/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:54:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:54:06.639046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf571dc7-d83f-46a3-bff0-00f0b5ce69de None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:06.639613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf571dc7-d83f-46a3-bff0-00f0b5ce69de None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:06.639613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf571dc7-d83f-46a3-bff0-00f0b5ce69de None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:06.641065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 748/1491] 10.222.0.40 () {54 vars in 926 bytes} [Mon Jul 25 05:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:54:06.646295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978fc8e-b7ca-42ac-b3de-af042f3d32d5 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:06.646719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978fc8e-b7ca-42ac-b3de-af042f3d32d5 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:06.646881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978fc8e-b7ca-42ac-b3de-af042f3d32d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:06.710050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3978fc8e-b7ca-42ac-b3de-af042f3d32d5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:54:06.711371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3978fc8e-b7ca-42ac-b3de-af042f3d32d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:06.806784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 744/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Mon Jul 25 05:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:54:06.845901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf571dc7-d83f-46a3-bff0-00f0b5ce69de None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:54:06.847436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cf571dc7-d83f-46a3-bff0-00f0b5ce69de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:06.955015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf571dc7-d83f-46a3-bff0-00f0b5ce69de None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:54:07.028500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d6a89-ced3-4626-8292-52c06f8c81c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:54:07.029246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d6a89-ced3-4626-8292-52c06f8c81c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:54:07.029528 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d6a89-ced3-4626-8292-52c06f8c81c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:54:07.029801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d6a89-ced3-4626-8292-52c06f8c81c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:54:07.031447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cf0d6a89-ced3-4626-8292-52c06f8c81c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:54:07.040317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:07.139151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d6a89-ced3-4626-8292-52c06f8c81c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:54:07.140634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d6a89-ced3-4626-8292-52c06f8c81c5 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:54:07.158921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 749/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:54:16.804945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=82 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:54:17.156745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=50 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:42.085469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeaa90-f2b8-4613-9742-935cf7b4bc89 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:42.086037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeaa90-f2b8-4613-9742-935cf7b4bc89 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:42.086037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeaa90-f2b8-4613-9742-935cf7b4bc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:42.158129 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-dcaeaa90-f2b8-4613-9742-935cf7b4bc89 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:55:42.159335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-dcaeaa90-f2b8-4613-9742-935cf7b4bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:42.256483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 745/1494] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:55:42.265368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1e99a-a9ab-448b-8dbd-6c1a15bb3692 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:42.265883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1e99a-a9ab-448b-8dbd-6c1a15bb3692 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:42.266003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1e99a-a9ab-448b-8dbd-6c1a15bb3692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:42.338506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-97d1e99a-a9ab-448b-8dbd-6c1a15bb3692 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:55:42.339619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-97d1e99a-a9ab-448b-8dbd-6c1a15bb3692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:42.433207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 750/1495] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:55:42.438719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaeaa90-f2b8-4613-9742-935cf7b4bc89 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:42.439933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-dcaeaa90-f2b8-4613-9742-935cf7b4bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:42.451473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaeaa90-f2b8-4613-9742-935cf7b4bc89 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:42.515340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:42.548975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:42.549641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:42.549923 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:42.550199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:42.550916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:55:42.551243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:42.552310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING py.warnings [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:55:42.552310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: warnings.warn(msg) Jul 25 05:55:42.552310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: Jul 25 05:55:42.553191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:42.565510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 746/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:55:42.573569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1e99a-a9ab-448b-8dbd-6c1a15bb3692 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:42.582711 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1e99a-a9ab-448b-8dbd-6c1a15bb3692 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:42.643199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:42.676468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:42.677131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:42.677414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:42.677690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:42.678386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:42.680099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING py.warnings [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:55:42.680099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: warnings.warn(msg) Jul 25 05:55:42.680099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: Jul 25 05:55:42.681298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:42.795459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 751/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:55:42 2022] POST /identity/v3/projects => generated 379 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:55:42.799666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:42.801285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:42.833500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74b3e6-4e89-4a91-a629-bd10a571cff6 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:42.929084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:42.929883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:42.930170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:42.930447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:42.931257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:42.932317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING py.warnings [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:55:42.932317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: warnings.warn(msg) Jul 25 05:55:42.932317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: Jul 25 05:55:42.933557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:43.019885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 747/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:55:42 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:55:43.024020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:43.025689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:43.088750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63792a0-ce42-4ffe-86c6-19530df87ed7 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:43.172999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:43.185546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:43.185546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:43.185546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:43.185546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:43.186172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:43.187180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING py.warnings [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:55:43.187180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: warnings.warn(msg) Jul 25 05:55:43.187180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: Jul 25 05:55:43.188054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:43.201238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 752/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:55:43.205261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:43.206862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:43.219234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470d057-569f-491b-940f-c5cb79a000a3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:43.243964 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:43.314025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3a189-4c41-4355-8243-927f894183fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:43.314693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a189-4c41-4355-8243-927f894183fb admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:43.314977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a189-4c41-4355-8243-927f894183fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:43.315252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a189-4c41-4355-8243-927f894183fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:43.315974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3a189-4c41-4355-8243-927f894183fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:43.317013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING py.warnings [None req-c0d3a189-4c41-4355-8243-927f894183fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:55:43.317013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: warnings.warn(msg) Jul 25 05:55:43.317013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: Jul 25 05:55:43.317897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3a189-4c41-4355-8243-927f894183fb admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:43.329980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 748/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:55:43.334183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:43.335808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:43.369153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071f157-e86a-4593-9ad0-72271991ae39 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:43.462448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2e360-efb0-47d6-b61b-4b395f5b974e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:43.463257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2e360-efb0-47d6-b61b-4b395f5b974e admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:43.463541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2e360-efb0-47d6-b61b-4b395f5b974e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:43.463819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2e360-efb0-47d6-b61b-4b395f5b974e admin admin] PATH_INFO: `/v3/projects/64e4ed1a3de6499e9a4affd0e0866ce4/users/adf7da3edbf44c6a9e7bd63836140112/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:43.486830 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2e360-efb0-47d6-b61b-4b395f5b974e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e4ed1a3de6499e9a4affd0e0866ce4, user_id=adf7da3edbf44c6a9e7bd63836140112, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:43.487853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING py.warnings [None req-3ae2e360-efb0-47d6-b61b-4b395f5b974e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:55:43.487853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: warnings.warn(msg) Jul 25 05:55:43.487853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: Jul 25 05:55:43.489799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2e360-efb0-47d6-b61b-4b395f5b974e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:43.568549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 753/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:55:43 2022] PUT /identity/v3/projects/64e4ed1a3de6499e9a4affd0e0866ce4/users/adf7da3edbf44c6a9e7bd63836140112/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:55:43.580876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e61192-4dce-43d2-93c1-e6dcc556565b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:43.581188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e61192-4dce-43d2-93c1-e6dcc556565b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:43.581456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e61192-4dce-43d2-93c1-e6dcc556565b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:43.636532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-40e61192-4dce-43d2-93c1-e6dcc556565b None None] MFA Rules not processed for user `adf7da3edbf44c6a9e7bd63836140112`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:55:43.637724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-40e61192-4dce-43d2-93c1-e6dcc556565b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:43.715495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 749/1502] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:55:43.728435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa1fe9-25a5-4875-84b8-618175adb5d7 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:43.728730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa1fe9-25a5-4875-84b8-618175adb5d7 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:43.729015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa1fe9-25a5-4875-84b8-618175adb5d7 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:43.730396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 754/1503] 10.222.0.40 () {54 vars in 964 bytes} [Mon Jul 25 05:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:55:43.736052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f90c6c-f8f5-479e-bb7c-b8f6a75eacfc None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:43.736373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f90c6c-f8f5-479e-bb7c-b8f6a75eacfc None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:43.736652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f90c6c-f8f5-479e-bb7c-b8f6a75eacfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:43.766378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:43.811184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-26f90c6c-f8f5-479e-bb7c-b8f6a75eacfc None None] MFA Rules not processed for user `f8a47b60be1a45f9aabc262b2d04d596`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:55:43.812427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-26f90c6c-f8f5-479e-bb7c-b8f6a75eacfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:43.886472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 750/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Mon Jul 25 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:55:43.894784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa1fe9-25a5-4875-84b8-618175adb5d7 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:43.896353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-09fa1fe9-25a5-4875-84b8-618175adb5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:43.946514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:44.000477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa1fe9-25a5-4875-84b8-618175adb5d7 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:44.073662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:44.074803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:44.075086 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:44.075362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:44.077054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:44.175062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:44.176788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:44.180286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:44.194869 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 755/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:55:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:55:44.237604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e252-7052-4b1f-b0c4-0176d6a83b6e None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:44.238010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e252-7052-4b1f-b0c4-0176d6a83b6e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:44.238158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939e252-7052-4b1f-b0c4-0176d6a83b6e None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:44.239458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 751/1506] 10.222.0.40 () {52 vars in 829 bytes} [Mon Jul 25 05:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:55:44.242938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:44.244286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:44.342545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29390d1b-0b18-4f88-b7b4-4ac502f927cf service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:44.436027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:44.436935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:44.437218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:44.437497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] PATH_INFO: `/v3/users/adf7da3edbf44c6a9e7bd63836140112` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:44.450500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=adf7da3edbf44c6a9e7bd63836140112)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:44.451529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING py.warnings [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:55:44.451529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: warnings.warn(msg) Jul 25 05:55:44.451529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: Jul 25 05:55:44.452797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:44.462476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:44.649262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adf7da3edbf44c6a9e7bd63836140112'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:44.722159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adf7da3edbf44c6a9e7bd63836140112'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:44.722598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adf7da3edbf44c6a9e7bd63836140112'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:44.734918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adf7da3edbf44c6a9e7bd63836140112'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:44.735459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adf7da3edbf44c6a9e7bd63836140112'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:44.758722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 756/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 05:55:44 2022] DELETE /identity/v3/users/adf7da3edbf44c6a9e7bd63836140112 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:55:44.764142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49b797-1fac-47cc-ae65-09cbaa6c2857 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:44.764430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49b797-1fac-47cc-ae65-09cbaa6c2857 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:44.764708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49b797-1fac-47cc-ae65-09cbaa6c2857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:44.821088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-0c49b797-1fac-47cc-ae65-09cbaa6c2857 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:55:44.822330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0c49b797-1fac-47cc-ae65-09cbaa6c2857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:44.899179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:44.912649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 752/1508] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:55:44.922256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:44.923563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:45.023552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076206eb-5f69-431c-bcc7-4a7eade5fd0c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:45.078569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:45.097081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14741027-4aa5-4ff1-927a-f99b1c275720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:45.097732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14741027-4aa5-4ff1-927a-f99b1c275720 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:45.098015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14741027-4aa5-4ff1-927a-f99b1c275720 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:45.098295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14741027-4aa5-4ff1-927a-f99b1c275720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:45.099997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-14741027-4aa5-4ff1-927a-f99b1c275720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:45.211507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14741027-4aa5-4ff1-927a-f99b1c275720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:45.213106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14741027-4aa5-4ff1-927a-f99b1c275720 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:45.234865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 757/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:55:45.782198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49b797-1fac-47cc-ae65-09cbaa6c2857 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:55:45.783727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0c49b797-1fac-47cc-ae65-09cbaa6c2857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:55:45.796095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49b797-1fac-47cc-ae65-09cbaa6c2857 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:55:45.892803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:55:45.893668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:55:45.893949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:55:45.894226 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] PATH_INFO: `/v3/projects/64e4ed1a3de6499e9a4affd0e0866ce4` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:55:45.906061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e4ed1a3de6499e9a4affd0e0866ce4)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:55:45.907080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING py.warnings [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:55:45.907080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: warnings.warn(msg) Jul 25 05:55:45.907080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: Jul 25 05:55:45.908304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:55:46.041106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e4ed1a3de6499e9a4affd0e0866ce4', 'request_id': 'req-e7e5b1d2-29e7-4bff-9574-744c41bb181d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:46.134079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e4ed1a3de6499e9a4affd0e0866ce4', 'request_id': 'req-e7e5b1d2-29e7-4bff-9574-744c41bb181d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:46.141356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e4ed1a3de6499e9a4affd0e0866ce4', 'request_id': 'req-e7e5b1d2-29e7-4bff-9574-744c41bb181d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:46.172239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] The token cache is being invalidate because project 64e4ed1a3de6499e9a4affd0e0866ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:55:46.172525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:46.172992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e7e5b1d2-29e7-4bff-9574-744c41bb181d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:55:46.174446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 753/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 05:55:45 2022] DELETE /identity/v3/projects/64e4ed1a3de6499e9a4affd0e0866ce4 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:55:55.232548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=62 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:55:56.170248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:57:23.353517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4f6a4-5539-4c0b-9455-bc9459ab5e39 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:23.354147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4f6a4-5539-4c0b-9455-bc9459ab5e39 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:23.354147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4f6a4-5539-4c0b-9455-bc9459ab5e39 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:23.355469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:23.360481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1131e-c969-4999-83d2-df3e1318fd37 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:23.360926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1131e-c969-4999-83d2-df3e1318fd37 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:23.361099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c1131e-c969-4999-83d2-df3e1318fd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:23.440337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c2c1131e-c969-4999-83d2-df3e1318fd37 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:23.441537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c2c1131e-c969-4999-83d2-df3e1318fd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:23.543514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:23.699669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4f6a4-5539-4c0b-9455-bc9459ab5e39 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:23.701836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9ea4f6a4-5539-4c0b-9455-bc9459ab5e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:23.821860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4f6a4-5539-4c0b-9455-bc9459ab5e39 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:23.827480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6faf8f-b808-4605-99cf-62f2147491a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:23.828132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6faf8f-b808-4605-99cf-62f2147491a1 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:23.828433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6faf8f-b808-4605-99cf-62f2147491a1 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:23.828708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6faf8f-b808-4605-99cf-62f2147491a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:23.836642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6faf8f-b808-4605-99cf-62f2147491a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:23.838210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6faf8f-b808-4605-99cf-62f2147491a1 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:23.848353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 759/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:24.690069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137aa93-400d-4aea-8214-a1ad398e7174 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:24.690624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137aa93-400d-4aea-8214-a1ad398e7174 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:24.690624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137aa93-400d-4aea-8214-a1ad398e7174 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:24.692070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:24.697357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21183eb-b6e4-42c7-aba5-b9880fbaaf71 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:24.697808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21183eb-b6e4-42c7-aba5-b9880fbaaf71 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:24.697973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21183eb-b6e4-42c7-aba5-b9880fbaaf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:24.716728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-a21183eb-b6e4-42c7-aba5-b9880fbaaf71 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:24.717979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a21183eb-b6e4-42c7-aba5-b9880fbaaf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:24.725989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:24.880785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137aa93-400d-4aea-8214-a1ad398e7174 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:24.895107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137aa93-400d-4aea-8214-a1ad398e7174 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:24.900560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91fcf6-e1c6-4864-bf89-e7654ee5fb25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:24.901199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fcf6-e1c6-4864-bf89-e7654ee5fb25 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:24.902015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fcf6-e1c6-4864-bf89-e7654ee5fb25 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:24.902324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91fcf6-e1c6-4864-bf89-e7654ee5fb25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:24.910613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91fcf6-e1c6-4864-bf89-e7654ee5fb25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:24.912240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91fcf6-e1c6-4864-bf89-e7654ee5fb25 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:24.922225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 756/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:25.193605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82523fa-5f77-49f4-80fd-126547fb95b3 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:25.193903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82523fa-5f77-49f4-80fd-126547fb95b3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:25.194190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82523fa-5f77-49f4-80fd-126547fb95b3 None None] PATH_INFO: `/v3` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:25.195543 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 761/1517] 10.222.0.47 () {54 vars in 925 bytes} [Mon Jul 25 05:57:25 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:57:25.201881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4693-3b86-46d6-8c96-d03dfc34dbbd None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:25.202175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4693-3b86-46d6-8c96-d03dfc34dbbd None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:25.202459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4693-3b86-46d6-8c96-d03dfc34dbbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:25.259111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1eed4693-3b86-46d6-8c96-d03dfc34dbbd None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:25.260455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1eed4693-3b86-46d6-8c96-d03dfc34dbbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:25.278454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:57:25.334907 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 757/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Jul 25 05:57:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:57:29.280329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b62ba-2951-4373-9538-f096b0a121f2 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:29.280862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b62ba-2951-4373-9538-f096b0a121f2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:29.280862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b62ba-2951-4373-9538-f096b0a121f2 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:29.282250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 762/1519] 10.222.0.40 () {54 vars in 961 bytes} [Mon Jul 25 05:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:29.287472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8649f-7f35-4eaf-92ec-538f0d062ac3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:29.287895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8649f-7f35-4eaf-92ec-538f0d062ac3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:29.288034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8649f-7f35-4eaf-92ec-538f0d062ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:29.306421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-42f8649f-7f35-4eaf-92ec-538f0d062ac3 None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:29.308205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-42f8649f-7f35-4eaf-92ec-538f0d062ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:29.317244 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 758/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Jul 25 05:57:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:29.325630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b62ba-2951-4373-9538-f096b0a121f2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:29.339010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b62ba-2951-4373-9538-f096b0a121f2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:29.344232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e5bbd-8918-467e-a92f-a3bb894effed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:29.344876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5bbd-8918-467e-a92f-a3bb894effed service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:29.345155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5bbd-8918-467e-a92f-a3bb894effed service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:29.345431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e5bbd-8918-467e-a92f-a3bb894effed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:29.353795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e5bbd-8918-467e-a92f-a3bb894effed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:29.358081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e5bbd-8918-467e-a92f-a3bb894effed service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:29.365169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 763/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:36.170279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d05a-7e1a-480a-8bb2-8426c59a8302 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:36.170872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d05a-7e1a-480a-8bb2-8426c59a8302 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:36.170872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067d05a-7e1a-480a-8bb2-8426c59a8302 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:36.172220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:36.177094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818e1ab-904c-43cc-b946-3e785b2d7ea5 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:36.177529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818e1ab-904c-43cc-b946-3e785b2d7ea5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:36.177675 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818e1ab-904c-43cc-b946-3e785b2d7ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:36.195884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2818e1ab-904c-43cc-b946-3e785b2d7ea5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:36.197124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2818e1ab-904c-43cc-b946-3e785b2d7ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:36.205069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:36.357143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067d05a-7e1a-480a-8bb2-8426c59a8302 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:36.371927 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067d05a-7e1a-480a-8bb2-8426c59a8302 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:36.377416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceacfe2-6afb-462b-83a3-17d13c0bfb1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:36.378052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceacfe2-6afb-462b-83a3-17d13c0bfb1b service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:36.378335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceacfe2-6afb-462b-83a3-17d13c0bfb1b service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:36.378610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceacfe2-6afb-462b-83a3-17d13c0bfb1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:36.386850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceacfe2-6afb-462b-83a3-17d13c0bfb1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:36.388496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceacfe2-6afb-462b-83a3-17d13c0bfb1b service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:36.397806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 760/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:37.238606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16db3-15f7-4842-aadf-ae36501cecc2 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:37.239190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16db3-15f7-4842-aadf-ae36501cecc2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:37.239190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca16db3-15f7-4842-aadf-ae36501cecc2 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:37.240594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 25 05:57:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:37.245273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce1bb-eb07-42a5-877a-fe7727bb54ee None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:37.245682 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce1bb-eb07-42a5-877a-fe7727bb54ee None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:37.246012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce1bb-eb07-42a5-877a-fe7727bb54ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:37.265099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-845ce1bb-eb07-42a5-877a-fe7727bb54ee None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:37.266321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-845ce1bb-eb07-42a5-877a-fe7727bb54ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:37.274363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 25 05:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:37.427795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca16db3-15f7-4842-aadf-ae36501cecc2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:37.442481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca16db3-15f7-4842-aadf-ae36501cecc2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:37.447803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d01ce-8249-432b-bebf-39e0fbfe5871 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:37.448468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d01ce-8249-432b-bebf-39e0fbfe5871 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:37.448782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d01ce-8249-432b-bebf-39e0fbfe5871 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:37.449628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d01ce-8249-432b-bebf-39e0fbfe5871 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:37.457590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d01ce-8249-432b-bebf-39e0fbfe5871 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:37.459150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d01ce-8249-432b-bebf-39e0fbfe5871 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:37.468389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 766/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:41.405856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f6f83-f5b9-45a5-abcc-3a6783fc4930 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:41.406429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f6f83-f5b9-45a5-abcc-3a6783fc4930 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:41.406429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f6f83-f5b9-45a5-abcc-3a6783fc4930 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:41.408046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:41.413226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d03b8-fa64-46c9-9807-20e97d29eeca None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:41.413643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d03b8-fa64-46c9-9807-20e97d29eeca None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:41.413791 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d03b8-fa64-46c9-9807-20e97d29eeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:41.432571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b74d03b8-fa64-46c9-9807-20e97d29eeca None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:41.434396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b74d03b8-fa64-46c9-9807-20e97d29eeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:41.442312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:41.518567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f6f83-f5b9-45a5-abcc-3a6783fc4930 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:41.520291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-da1f6f83-f5b9-45a5-abcc-3a6783fc4930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:41.599299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f6f83-f5b9-45a5-abcc-3a6783fc4930 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:41.602837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa9a79-a666-4f51-a1d5-676e4342d3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:41.603479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa9a79-a666-4f51-a1d5-676e4342d3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:41.603763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa9a79-a666-4f51-a1d5-676e4342d3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:41.604037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa9a79-a666-4f51-a1d5-676e4342d3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:41.613431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa9a79-a666-4f51-a1d5-676e4342d3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:41.617831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa9a79-a666-4f51-a1d5-676e4342d3e0 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:41.624905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 763/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:43.723312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a27b4-1c14-4538-a23a-189826dc9eac None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:43.723889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a27b4-1c14-4538-a23a-189826dc9eac None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:43.723889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a27b4-1c14-4538-a23a-189826dc9eac None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:43.725314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:43.730999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c42c00-7273-4419-af0c-912917b78186 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:43.731418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c42c00-7273-4419-af0c-912917b78186 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:43.731600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c42c00-7273-4419-af0c-912917b78186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:43.750062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-93c42c00-7273-4419-af0c-912917b78186 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:43.751280 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-93c42c00-7273-4419-af0c-912917b78186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:43.759407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:57:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:43.796039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a27b4-1c14-4538-a23a-189826dc9eac None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:43.806677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a27b4-1c14-4538-a23a-189826dc9eac None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:43.810223 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed2b0e-7de5-45b8-a33c-6c6f16c5bd8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:43.810854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2b0e-7de5-45b8-a33c-6c6f16c5bd8c service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:43.811136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2b0e-7de5-45b8-a33c-6c6f16c5bd8c service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:43.811411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed2b0e-7de5-45b8-a33c-6c6f16c5bd8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:43.820216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2b0e-7de5-45b8-a33c-6c6f16c5bd8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:43.821642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed2b0e-7de5-45b8-a33c-6c6f16c5bd8c service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:43.831622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 769/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:57:47.626940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac36b47-c968-4684-9b37-e7da314aeb7f None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:47.627513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac36b47-c968-4684-9b37-e7da314aeb7f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:47.627513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac36b47-c968-4684-9b37-e7da314aeb7f None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:47.628956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:47.634196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7ce63-26c1-4d7e-90d4-6624b9e0e389 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:47.634631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7ce63-26c1-4d7e-90d4-6624b9e0e389 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:47.634759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7ce63-26c1-4d7e-90d4-6624b9e0e389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:47.642012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=81 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:57:47.654626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-91d7ce63-26c1-4d7e-90d4-6624b9e0e389 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:47.655830 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-91d7ce63-26c1-4d7e-90d4-6624b9e0e389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:47.664170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:57:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:47.970005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac36b47-c968-4684-9b37-e7da314aeb7f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:47.971682 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fac36b47-c968-4684-9b37-e7da314aeb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:48.039028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:57:48.052192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac36b47-c968-4684-9b37-e7da314aeb7f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:48.055818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b698c-eb59-4e59-8861-ec5f9df4abdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:48.056529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b698c-eb59-4e59-8861-ec5f9df4abdc service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:48.056817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b698c-eb59-4e59-8861-ec5f9df4abdc service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:48.057097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b698c-eb59-4e59-8861-ec5f9df4abdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:48.067333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b698c-eb59-4e59-8861-ec5f9df4abdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:48.068807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b698c-eb59-4e59-8861-ec5f9df4abdc service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:48.079197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 766/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:57:50.145347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b80a5-da60-4afb-9aac-9763a9e348ba None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:50.145874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b80a5-da60-4afb-9aac-9763a9e348ba None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:50.145874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b80a5-da60-4afb-9aac-9763a9e348ba None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:50.147196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 25 05:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:57:50.153144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ace72-dce7-42f2-af1b-8f80dc94aa67 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:50.153599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ace72-dce7-42f2-af1b-8f80dc94aa67 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:50.153704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ace72-dce7-42f2-af1b-8f80dc94aa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:50.172584 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-925ace72-dce7-42f2-af1b-8f80dc94aa67 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:57:50.173785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-925ace72-dce7-42f2-af1b-8f80dc94aa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:50.181670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 25 05:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:57:50.485050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b80a5-da60-4afb-9aac-9763a9e348ba None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:57:50.486743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d60b80a5-da60-4afb-9aac-9763a9e348ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:57:50.500173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b80a5-da60-4afb-9aac-9763a9e348ba None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:57:50.503690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ba3621-fdb9-4418-a2b7-50e38bce7d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:57:50.504355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba3621-fdb9-4418-a2b7-50e38bce7d7f service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:57:50.504639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba3621-fdb9-4418-a2b7-50e38bce7d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:57:50.504916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ba3621-fdb9-4418-a2b7-50e38bce7d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:57:50.514459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba3621-fdb9-4418-a2b7-50e38bce7d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:57:50.515899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ba3621-fdb9-4418-a2b7-50e38bce7d7f service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:57:50.525113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 772/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:58:00.166796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=11 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:58:00.521143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=8 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:18.520958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88ccd3-ae19-47d7-ba65-6e2fa4b3df1e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:18.521540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88ccd3-ae19-47d7-ba65-6e2fa4b3df1e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:18.521540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88ccd3-ae19-47d7-ba65-6e2fa4b3df1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:18.548911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-6c88ccd3-ae19-47d7-ba65-6e2fa4b3df1e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:18.550838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-6c88ccd3-ae19-47d7-ba65-6e2fa4b3df1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:18.564352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 768/1540] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:18.572908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e524-e283-4ec1-ae8f-4374f77566a6 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:18.574501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e524-e283-4ec1-ae8f-4374f77566a6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:18.574984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6e524-e283-4ec1-ae8f-4374f77566a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:18.600887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-33e6e524-e283-4ec1-ae8f-4374f77566a6 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:18.602444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-33e6e524-e283-4ec1-ae8f-4374f77566a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:18.613701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 773/1541] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:18.619541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88ccd3-ae19-47d7-ba65-6e2fa4b3df1e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:18.633805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88ccd3-ae19-47d7-ba65-6e2fa4b3df1e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:18.643193 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:18.643841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:18.644166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:18.644492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:18.645399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:18.645716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:18.647346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:18.665251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 769/1542] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:18.674732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6e524-e283-4ec1-ae8f-4374f77566a6 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:18.690466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6e524-e283-4ec1-ae8f-4374f77566a6 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:18.698409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f4412-0a90-4bdd-b1fe-64ddb0e0c56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:18.699959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f4412-0a90-4bdd-b1fe-64ddb0e0c56c admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:18.699959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f4412-0a90-4bdd-b1fe-64ddb0e0c56c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:18.699959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f4412-0a90-4bdd-b1fe-64ddb0e0c56c admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:18.700733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f4412-0a90-4bdd-b1fe-64ddb0e0c56c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:18.702846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f4412-0a90-4bdd-b1fe-64ddb0e0c56c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:18.813529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 774/1543] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:18 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:18.818121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:18.830904 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd739c-190b-4042-bd5a-bb39e406a1b4 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:18.845386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e222a-a3f5-42b1-baeb-d57d1e318075 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:18.845734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e222a-a3f5-42b1-baeb-d57d1e318075 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:18.845943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e222a-a3f5-42b1-baeb-d57d1e318075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:18.875699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-708e222a-a3f5-42b1-baeb-d57d1e318075 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:18.881051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-708e222a-a3f5-42b1-baeb-d57d1e318075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:18.974773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050e620-5029-48d4-bc93-9e8586603802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:18.975462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050e620-5029-48d4-bc93-9e8586603802 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:18.975748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050e620-5029-48d4-bc93-9e8586603802 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:18.976033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050e620-5029-48d4-bc93-9e8586603802 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:18.976791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050e620-5029-48d4-bc93-9e8586603802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:18.978755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050e620-5029-48d4-bc93-9e8586603802 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.019449 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 775/1544] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:19.028919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d332ec-9fdb-40ed-9fac-8f07c6a4b540 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.029217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d332ec-9fdb-40ed-9fac-8f07c6a4b540 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.029491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d332ec-9fdb-40ed-9fac-8f07c6a4b540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.054063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-88d332ec-9fdb-40ed-9fac-8f07c6a4b540 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.055275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-88d332ec-9fdb-40ed-9fac-8f07c6a4b540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.067737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 776/1545] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:19.074852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d332ec-9fdb-40ed-9fac-8f07c6a4b540 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.085042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d332ec-9fdb-40ed-9fac-8f07c6a4b540 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.093127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.093809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.094095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.094372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.095119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:19.095425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.096479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING py.warnings [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:59:19.096479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: warnings.warn(msg) Jul 25 05:59:19.096479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: Jul 25 05:59:19.100769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.111908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 770/1546] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:18 2022] POST /identity/v3/users => generated 288 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:19.113471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53b3b0-4f7e-42c9-b9e4-7ae23a75f251 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.113771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53b3b0-4f7e-42c9-b9e4-7ae23a75f251 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.114053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53b3b0-4f7e-42c9-b9e4-7ae23a75f251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.115617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 777/1547] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:59:19.117426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.129538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66487f6-f022-4664-adfe-9b507952eb1b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.149534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c836d7-a9fe-45d2-ae48-47d8414c0b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.150232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c836d7-a9fe-45d2-ae48-47d8414c0b62 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.150478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1f53b3b0-4f7e-42c9-b9e4-7ae23a75f251 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.150600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c836d7-a9fe-45d2-ae48-47d8414c0b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.150768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c836d7-a9fe-45d2-ae48-47d8414c0b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.151514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c836d7-a9fe-45d2-ae48-47d8414c0b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.151698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1f53b3b0-4f7e-42c9-b9e4-7ae23a75f251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.154342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c836d7-a9fe-45d2-ae48-47d8414c0b62 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.170019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 771/1548] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.171574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53b3b0-4f7e-42c9-b9e4-7ae23a75f251 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.178552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 778/1549] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:19.180010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289d56-856f-40a9-b5cc-cce83eecdff4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.180329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289d56-856f-40a9-b5cc-cce83eecdff4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.180614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71289d56-856f-40a9-b5cc-cce83eecdff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.189768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53b3b0-4f7e-42c9-b9e4-7ae23a75f251 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.208373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd30c08-5b60-48c4-b9da-dcf5be7e1963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.209064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd30c08-5b60-48c4-b9da-dcf5be7e1963 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.209420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd30c08-5b60-48c4-b9da-dcf5be7e1963 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.209696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd30c08-5b60-48c4-b9da-dcf5be7e1963 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.210398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd30c08-5b60-48c4-b9da-dcf5be7e1963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.211422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING py.warnings [None req-0fd30c08-5b60-48c4-b9da-dcf5be7e1963 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:59:19.211422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: warnings.warn(msg) Jul 25 05:59:19.211422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: Jul 25 05:59:19.212664 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd30c08-5b60-48c4-b9da-dcf5be7e1963 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.215784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-71289d56-856f-40a9-b5cc-cce83eecdff4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.217500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-71289d56-856f-40a9-b5cc-cce83eecdff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.230779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 779/1550] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.232104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94731f-07a9-49c1-b5b2-0025d37307b2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.232421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94731f-07a9-49c1-b5b2-0025d37307b2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.232707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94731f-07a9-49c1-b5b2-0025d37307b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.256714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-bd94731f-07a9-49c1-b5b2-0025d37307b2 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.257995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bd94731f-07a9-49c1-b5b2-0025d37307b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.266711 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 780/1551] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.267473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94731f-07a9-49c1-b5b2-0025d37307b2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.276623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd94731f-07a9-49c1-b5b2-0025d37307b2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.281854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b06cd2-3115-4ba2-9c54-7849e2a8aef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.282495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b06cd2-3115-4ba2-9c54-7849e2a8aef9 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.282780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b06cd2-3115-4ba2-9c54-7849e2a8aef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.283050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b06cd2-3115-4ba2-9c54-7849e2a8aef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.283823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b06cd2-3115-4ba2-9c54-7849e2a8aef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.285438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b06cd2-3115-4ba2-9c54-7849e2a8aef9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.295357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 781/1552] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:19.296533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c364701-80fd-4030-a07f-c066d8c9f17f None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.296818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c364701-80fd-4030-a07f-c066d8c9f17f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.297094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c364701-80fd-4030-a07f-c066d8c9f17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.314264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9c364701-80fd-4030-a07f-c066d8c9f17f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.315438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9c364701-80fd-4030-a07f-c066d8c9f17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.323234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 782/1553] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.325736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af5b9-fd20-4b0a-8d63-08480f5d24fb None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.328581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af5b9-fd20-4b0a-8d63-08480f5d24fb None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.328581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3af5b9-fd20-4b0a-8d63-08480f5d24fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.336520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 772/1554] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:19.336520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7bea5-19d3-4216-bd34-3c633d65a376 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.336520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7bea5-19d3-4216-bd34-3c633d65a376 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.336520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e7bea5-19d3-4216-bd34-3c633d65a376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.345149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-6f3af5b9-fd20-4b0a-8d63-08480f5d24fb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.346377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6f3af5b9-fd20-4b0a-8d63-08480f5d24fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.353819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d1e7bea5-19d3-4216-bd34-3c633d65a376 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.357232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d1e7bea5-19d3-4216-bd34-3c633d65a376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.405736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 783/1555] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.407062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f852a55-40ff-4062-a62a-d514e7be6155 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.407357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f852a55-40ff-4062-a62a-d514e7be6155 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.407642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f852a55-40ff-4062-a62a-d514e7be6155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.413977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:19.420598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 773/1556] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.425328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506ea3-a6fe-4a14-acab-80ba77a3bd06 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.425328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506ea3-a6fe-4a14-acab-80ba77a3bd06 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.425328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63506ea3-a6fe-4a14-acab-80ba77a3bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.426669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-8f852a55-40ff-4062-a62a-d514e7be6155 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.427873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8f852a55-40ff-4062-a62a-d514e7be6155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.436305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 784/1557] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.437105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f852a55-40ff-4062-a62a-d514e7be6155 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.442714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-63506ea3-a6fe-4a14-acab-80ba77a3bd06 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.444334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-63506ea3-a6fe-4a14-acab-80ba77a3bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.453168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f852a55-40ff-4062-a62a-d514e7be6155 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.453326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 774/1558] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.454022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63506ea3-a6fe-4a14-acab-80ba77a3bd06 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.463477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63506ea3-a6fe-4a14-acab-80ba77a3bd06 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.464373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c7f26-565c-40b4-8972-c93ddfb438d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.466026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c7f26-565c-40b4-8972-c93ddfb438d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.466716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c7f26-565c-40b4-8972-c93ddfb438d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.467344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c7f26-565c-40b4-8972-c93ddfb438d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.468767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bce16e-1aad-4751-ad65-6c67c98dbf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.468951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c7f26-565c-40b4-8972-c93ddfb438d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:19.469270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c7f26-565c-40b4-8972-c93ddfb438d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.469643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bce16e-1aad-4751-ad65-6c67c98dbf97 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.469926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bce16e-1aad-4751-ad65-6c67c98dbf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.470200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bce16e-1aad-4751-ad65-6c67c98dbf97 admin admin] PATH_INFO: `/v3/projects/5ea60662f93343019fe380b6e13d5d18/users/a7224418acee432182bf3195bfc8fd99/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.470993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c7f26-565c-40b4-8972-c93ddfb438d8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.483319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 785/1559] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:19.484689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fadb7-5099-4cf2-bf4b-fb452660d24e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.484979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fadb7-5099-4cf2-bf4b-fb452660d24e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.485266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fadb7-5099-4cf2-bf4b-fb452660d24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.491766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bce16e-1aad-4751-ad65-6c67c98dbf97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea60662f93343019fe380b6e13d5d18, user_id=a7224418acee432182bf3195bfc8fd99, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.492859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING py.warnings [None req-13bce16e-1aad-4751-ad65-6c67c98dbf97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:59:19.492859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: warnings.warn(msg) Jul 25 05:59:19.492859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: Jul 25 05:59:19.494800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bce16e-1aad-4751-ad65-6c67c98dbf97 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.503600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-7f4fadb7-5099-4cf2-bf4b-fb452660d24e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.504823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7f4fadb7-5099-4cf2-bf4b-fb452660d24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.513348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 786/1560] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.517442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fadb7-5099-4cf2-bf4b-fb452660d24e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.523258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fadb7-5099-4cf2-bf4b-fb452660d24e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.528221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.528901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.529233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.529509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.530216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.531260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING py.warnings [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:59:19.531260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: warnings.warn(msg) Jul 25 05:59:19.531260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: Jul 25 05:59:19.532487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.605808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 775/1561] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:19 2022] PUT /identity/v3/projects/5ea60662f93343019fe380b6e13d5d18/users/a7224418acee432182bf3195bfc8fd99/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:19.607299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e3156-5964-4b64-a7b3-0357a9a766db None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.607596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e3156-5964-4b64-a7b3-0357a9a766db None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.607881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e3156-5964-4b64-a7b3-0357a9a766db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.626286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2c9e3156-5964-4b64-a7b3-0357a9a766db None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.627509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2c9e3156-5964-4b64-a7b3-0357a9a766db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.632894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:19.637734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 787/1562] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:19.637734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.645037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f2c06-d552-4a79-aca4-3765db5876b0 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.689029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 776/1563] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.690432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d1fc6-cb67-4aa1-a0e3-7974353d4fc7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.690694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d1fc6-cb67-4aa1-a0e3-7974353d4fc7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.690978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d1fc6-cb67-4aa1-a0e3-7974353d4fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.703442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a3551-257c-4020-b6de-98806a7142b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.708300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a3551-257c-4020-b6de-98806a7142b4 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.708300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a3551-257c-4020-b6de-98806a7142b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.708300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a3551-257c-4020-b6de-98806a7142b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.708300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a3551-257c-4020-b6de-98806a7142b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:19.708300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a3551-257c-4020-b6de-98806a7142b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.708300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a3551-257c-4020-b6de-98806a7142b4 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.710722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-030d1fc6-cb67-4aa1-a0e3-7974353d4fc7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.711967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-030d1fc6-cb67-4aa1-a0e3-7974353d4fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.728194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 788/1564] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:19.728402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 777/1565] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.728402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d1fc6-cb67-4aa1-a0e3-7974353d4fc7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.728591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9921b-fc50-45b6-89db-d73a32d69846 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.728591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9921b-fc50-45b6-89db-d73a32d69846 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.728591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9921b-fc50-45b6-89db-d73a32d69846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.732824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d1fc6-cb67-4aa1-a0e3-7974353d4fc7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.737426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d3f92-7579-4bd1-aee2-29af534535b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.738167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d3f92-7579-4bd1-aee2-29af534535b5 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.738390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d3f92-7579-4bd1-aee2-29af534535b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.738646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d3f92-7579-4bd1-aee2-29af534535b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.739339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d3f92-7579-4bd1-aee2-29af534535b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.741219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-49e9921b-fc50-45b6-89db-d73a32d69846 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.741333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d3f92-7579-4bd1-aee2-29af534535b5 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.742409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-49e9921b-fc50-45b6-89db-d73a32d69846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.750904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 789/1566] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.751685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9921b-fc50-45b6-89db-d73a32d69846 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.764511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9921b-fc50-45b6-89db-d73a32d69846 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.770159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc08039-c13e-4be6-883f-00e4a158e38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.770823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08039-c13e-4be6-883f-00e4a158e38d admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.771103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08039-c13e-4be6-883f-00e4a158e38d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.771377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc08039-c13e-4be6-883f-00e4a158e38d admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.772107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08039-c13e-4be6-883f-00e4a158e38d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:19.772438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08039-c13e-4be6-883f-00e4a158e38d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.774023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc08039-c13e-4be6-883f-00e4a158e38d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.794204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 790/1567] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:19.794204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bdf77-2134-4a5c-90b3-c39cfd063492 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.794204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bdf77-2134-4a5c-90b3-c39cfd063492 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.794204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3bdf77-2134-4a5c-90b3-c39cfd063492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.805676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fd3bdf77-2134-4a5c-90b3-c39cfd063492 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:19.806857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fd3bdf77-2134-4a5c-90b3-c39cfd063492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:19.812776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 778/1568] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:19.832808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d3f92-7579-4bd1-aee2-29af534535b5 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.832987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 791/1569] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:19.832987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bdf77-2134-4a5c-90b3-c39cfd063492 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.832987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3bdf77-2134-4a5c-90b3-c39cfd063492 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.877739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d3f92-7579-4bd1-aee2-29af534535b5 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.882376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2252628-e017-4715-b28c-24288948c0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.883045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2252628-e017-4715-b28c-24288948c0e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.883355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2252628-e017-4715-b28c-24288948c0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.883631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2252628-e017-4715-b28c-24288948c0e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.884385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2252628-e017-4715-b28c-24288948c0e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.885970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2252628-e017-4715-b28c-24288948c0e3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.887107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.887920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.888420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.889139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.889893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:19.890214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.892293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.896674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 779/1570] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:19.897548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2252628-e017-4715-b28c-24288948c0e3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.905267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 792/1571] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:19.906176 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.915503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2252628-e017-4715-b28c-24288948c0e3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.916933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6abc1-0c8c-4b62-a777-ea04e8902ac5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.921040 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.921695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.922112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.922231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d99c7a-fde1-44fb-970c-33e8ee82bc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.922391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.922798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d99c7a-fde1-44fb-970c-33e8ee82bc5f admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.923083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d99c7a-fde1-44fb-970c-33e8ee82bc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.923196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:19.923363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d99c7a-fde1-44fb-970c-33e8ee82bc5f admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.923472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.924048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d99c7a-fde1-44fb-970c-33e8ee82bc5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.925035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.925965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d99c7a-fde1-44fb-970c-33e8ee82bc5f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.938502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 780/1572] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:19.939381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.949878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df92e8-9d1c-44af-934c-c6a12d1dd42f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.954459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.965947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.965947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.965947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.965947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:19.965947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.965947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:19.969511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 781/1573] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:19.970385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:19.979662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4d1f4-227b-4d08-99c6-18b0aca8cfc2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:19.985066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:19.992710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:19.992710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:19.992710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:19.992710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:19.992710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.005549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:20.012885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 793/1574] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/projects => generated 393 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.013658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d99c7a-fde1-44fb-970c-33e8ee82bc5f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.077239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 782/1575] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:19 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.077239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.077611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d99c7a-fde1-44fb-970c-33e8ee82bc5f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.077734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:20.110672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:20.133347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f634656-c043-4dfb-91c5-14e5b304dfa1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.137624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e371a6-70d2-486a-a6da-7679a0e61c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.138276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e371a6-70d2-486a-a6da-7679a0e61c65 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.141994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e371a6-70d2-486a-a6da-7679a0e61c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.141994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e371a6-70d2-486a-a6da-7679a0e61c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.142189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c56f2-ea53-44ed-80db-7db2e3faeeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.142189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c56f2-ea53-44ed-80db-7db2e3faeeba admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.142189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c56f2-ea53-44ed-80db-7db2e3faeeba admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.142189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c56f2-ea53-44ed-80db-7db2e3faeeba admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.142555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c56f2-ea53-44ed-80db-7db2e3faeeba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.143730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c56f2-ea53-44ed-80db-7db2e3faeeba admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.147957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e371a6-70d2-486a-a6da-7679a0e61c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.149459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e371a6-70d2-486a-a6da-7679a0e61c65 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.159884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 783/1576] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:20.162004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e371a6-70d2-486a-a6da-7679a0e61c65 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.169936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e371a6-70d2-486a-a6da-7679a0e61c65 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.174571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3245260-8e69-4e92-b249-97250300921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.175278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3245260-8e69-4e92-b249-97250300921c admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.175568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3245260-8e69-4e92-b249-97250300921c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.175846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3245260-8e69-4e92-b249-97250300921c admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.176597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3245260-8e69-4e92-b249-97250300921c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.178183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3245260-8e69-4e92-b249-97250300921c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.189039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 784/1577] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:20.189823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3245260-8e69-4e92-b249-97250300921c admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.200205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3245260-8e69-4e92-b249-97250300921c admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.205163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fe293-dbf0-4d99-abde-b861d2d1d534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.205790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fe293-dbf0-4d99-abde-b861d2d1d534 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.206077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fe293-dbf0-4d99-abde-b861d2d1d534 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.206357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fe293-dbf0-4d99-abde-b861d2d1d534 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.207075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fe293-dbf0-4d99-abde-b861d2d1d534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.209063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fe293-dbf0-4d99-abde-b861d2d1d534 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.238884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 794/1578] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/users => generated 293 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.239696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c56f2-ea53-44ed-80db-7db2e3faeeba admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.249931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c56f2-ea53-44ed-80db-7db2e3faeeba admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.254501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae1d5a-f715-4614-b1c1-2beefff3ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.259042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae1d5a-f715-4614-b1c1-2beefff3ea8f admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.259379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae1d5a-f715-4614-b1c1-2beefff3ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.259672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae1d5a-f715-4614-b1c1-2beefff3ea8f admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.260419 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae1d5a-f715-4614-b1c1-2beefff3ea8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.263168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ae1d5a-f715-4614-b1c1-2beefff3ea8f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.315977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 785/1579] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.317131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fe293-dbf0-4d99-abde-b861d2d1d534 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.347763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 795/1580] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.347763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae1d5a-f715-4614-b1c1-2beefff3ea8f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.383287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fe293-dbf0-4d99-abde-b861d2d1d534 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.410766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae1d5a-f715-4614-b1c1-2beefff3ea8f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.415361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74740a3-eece-4563-a3b5-3b536b9a5b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.416064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74740a3-eece-4563-a3b5-3b536b9a5b84 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.416379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74740a3-eece-4563-a3b5-3b536b9a5b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.416767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74740a3-eece-4563-a3b5-3b536b9a5b84 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.417497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74740a3-eece-4563-a3b5-3b536b9a5b84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.420046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74740a3-eece-4563-a3b5-3b536b9a5b84 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.443005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87e283-2f72-4558-9263-416289a75012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.443728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e283-2f72-4558-9263-416289a75012 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.444049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e283-2f72-4558-9263-416289a75012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.444354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87e283-2f72-4558-9263-416289a75012 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.445061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87e283-2f72-4558-9263-416289a75012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.446986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87e283-2f72-4558-9263-416289a75012 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.529537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 796/1581] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.530428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74740a3-eece-4563-a3b5-3b536b9a5b84 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.535514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:20.541001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74740a3-eece-4563-a3b5-3b536b9a5b84 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.546686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bdba3-bef4-4065-b02b-16d408ce21ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.547358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bdba3-bef4-4065-b02b-16d408ce21ea admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.547643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bdba3-bef4-4065-b02b-16d408ce21ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.547921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277bdba3-bef4-4065-b02b-16d408ce21ea admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.548646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277bdba3-bef4-4065-b02b-16d408ce21ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.560917 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 786/1582] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.560917 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87e283-2f72-4558-9263-416289a75012 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.561155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277bdba3-bef4-4065-b02b-16d408ce21ea admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.585411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:20.613808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87e283-2f72-4558-9263-416289a75012 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.617601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 797/1583] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/users => generated 292 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.618592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bdba3-bef4-4065-b02b-16d408ce21ea admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.623370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad493bfb-1f5d-4e98-a866-7f11717b011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.624318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493bfb-1f5d-4e98-a866-7f11717b011e admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.624607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493bfb-1f5d-4e98-a866-7f11717b011e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.624913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad493bfb-1f5d-4e98-a866-7f11717b011e admin admin] PATH_INFO: `/v3/projects/e5643d28676f47869722468e73718165/users/c65f4282dd1e4f028d535b49febe12ee/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.629871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277bdba3-bef4-4065-b02b-16d408ce21ea admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.634647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed095f4-7735-44bb-a7ae-f1c7f6a0ed05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.635580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed095f4-7735-44bb-a7ae-f1c7f6a0ed05 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.635877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed095f4-7735-44bb-a7ae-f1c7f6a0ed05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.636988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed095f4-7735-44bb-a7ae-f1c7f6a0ed05 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.637829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed095f4-7735-44bb-a7ae-f1c7f6a0ed05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.639436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed095f4-7735-44bb-a7ae-f1c7f6a0ed05 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.641875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad493bfb-1f5d-4e98-a866-7f11717b011e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5643d28676f47869722468e73718165, user_id=c65f4282dd1e4f028d535b49febe12ee, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.644707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad493bfb-1f5d-4e98-a866-7f11717b011e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.650781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 798/1584] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:20.652021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed095f4-7735-44bb-a7ae-f1c7f6a0ed05 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.662284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed095f4-7735-44bb-a7ae-f1c7f6a0ed05 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.668643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd347d-5cab-4bdf-ae24-f5e98ebd0a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.669337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd347d-5cab-4bdf-ae24-f5e98ebd0a61 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.669608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd347d-5cab-4bdf-ae24-f5e98ebd0a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.669887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd347d-5cab-4bdf-ae24-f5e98ebd0a61 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.670578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd347d-5cab-4bdf-ae24-f5e98ebd0a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.672528 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd347d-5cab-4bdf-ae24-f5e98ebd0a61 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.802075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 799/1585] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.813539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 787/1586] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:20 2022] PUT /identity/v3/projects/e5643d28676f47869722468e73718165/users/c65f4282dd1e4f028d535b49febe12ee/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:20.813807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd347d-5cab-4bdf-ae24-f5e98ebd0a61 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.813991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad493bfb-1f5d-4e98-a866-7f11717b011e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.814754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd347d-5cab-4bdf-ae24-f5e98ebd0a61 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.884670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c440166-9da0-4ac9-b11b-5a1c62789d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.885143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad493bfb-1f5d-4e98-a866-7f11717b011e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.885292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c440166-9da0-4ac9-b11b-5a1c62789d44 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.885545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c440166-9da0-4ac9-b11b-5a1c62789d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.885824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c440166-9da0-4ac9-b11b-5a1c62789d44 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.886887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c440166-9da0-4ac9-b11b-5a1c62789d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.888979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c440166-9da0-4ac9-b11b-5a1c62789d44 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.892224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc405cb-7afa-4d9f-a4ff-cd865c854a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.892224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc405cb-7afa-4d9f-a4ff-cd865c854a49 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.892454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc405cb-7afa-4d9f-a4ff-cd865c854a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.892538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc405cb-7afa-4d9f-a4ff-cd865c854a49 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.893732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc405cb-7afa-4d9f-a4ff-cd865c854a49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.894986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc405cb-7afa-4d9f-a4ff-cd865c854a49 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.908366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 788/1587] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:20.909198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc405cb-7afa-4d9f-a4ff-cd865c854a49 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.919723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc405cb-7afa-4d9f-a4ff-cd865c854a49 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.924669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576f873-f217-491e-b6dd-5a4ea579a5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.925337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576f873-f217-491e-b6dd-5a4ea579a5ac admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.925628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576f873-f217-491e-b6dd-5a4ea579a5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.925909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576f873-f217-491e-b6dd-5a4ea579a5ac admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.926636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576f873-f217-491e-b6dd-5a4ea579a5ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.928684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576f873-f217-491e-b6dd-5a4ea579a5ac admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:20.970755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 800/1588] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:20.971790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c440166-9da0-4ac9-b11b-5a1c62789d44 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:20.982985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c440166-9da0-4ac9-b11b-5a1c62789d44 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:20.987641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74139d64-8c9c-458b-a52f-d094bf037dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:20.988334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74139d64-8c9c-458b-a52f-d094bf037dce admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:20.988620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74139d64-8c9c-458b-a52f-d094bf037dce admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:20.988913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74139d64-8c9c-458b-a52f-d094bf037dce admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:20.989625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74139d64-8c9c-458b-a52f-d094bf037dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:20.991268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74139d64-8c9c-458b-a52f-d094bf037dce admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.002709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 801/1589] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.003530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74139d64-8c9c-458b-a52f-d094bf037dce admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.014388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74139d64-8c9c-458b-a52f-d094bf037dce admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.018920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6b742-df23-4c0d-a166-e2f406ebba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.019567 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b742-df23-4c0d-a166-e2f406ebba22 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.019866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b742-df23-4c0d-a166-e2f406ebba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.020323 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b742-df23-4c0d-a166-e2f406ebba22 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.020984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6b742-df23-4c0d-a166-e2f406ebba22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.022588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6b742-df23-4c0d-a166-e2f406ebba22 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.030786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 789/1590] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:20 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:21.033022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576f873-f217-491e-b6dd-5a4ea579a5ac admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.035325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 802/1591] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.039548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d12b8-4dd3-402f-ac19-d254e30b6509 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.039852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d12b8-4dd3-402f-ac19-d254e30b6509 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.040139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d12b8-4dd3-402f-ac19-d254e30b6509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.043679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576f873-f217-491e-b6dd-5a4ea579a5ac admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.048628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0a66e-6f2f-4dc8-b941-4a4ea6228766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.049415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a66e-6f2f-4dc8-b941-4a4ea6228766 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.049715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a66e-6f2f-4dc8-b941-4a4ea6228766 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.049999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0a66e-6f2f-4dc8-b941-4a4ea6228766 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.050740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0a66e-6f2f-4dc8-b941-4a4ea6228766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.052409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0a66e-6f2f-4dc8-b941-4a4ea6228766 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.064744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 790/1592] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.065662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0a66e-6f2f-4dc8-b941-4a4ea6228766 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.071272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-598d12b8-4dd3-402f-ac19-d254e30b6509 None None] MFA Rules not processed for user `c65f4282dd1e4f028d535b49febe12ee`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:21.072638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-598d12b8-4dd3-402f-ac19-d254e30b6509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:21.078296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0a66e-6f2f-4dc8-b941-4a4ea6228766 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.083539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d278852-359b-441f-bfe3-d7da5810f2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.084373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d278852-359b-441f-bfe3-d7da5810f2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.084696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d278852-359b-441f-bfe3-d7da5810f2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.084960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d278852-359b-441f-bfe3-d7da5810f2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.085745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d278852-359b-441f-bfe3-d7da5810f2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.087370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d278852-359b-441f-bfe3-d7da5810f2a2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.098983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 791/1593] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.099796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d278852-359b-441f-bfe3-d7da5810f2a2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.112720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d278852-359b-441f-bfe3-d7da5810f2a2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.117432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fd0e3-af92-470d-970a-a2a6e4c04218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.120276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fd0e3-af92-470d-970a-a2a6e4c04218 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.120276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fd0e3-af92-470d-970a-a2a6e4c04218 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.120276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fd0e3-af92-470d-970a-a2a6e4c04218 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.120276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fd0e3-af92-470d-970a-a2a6e4c04218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.121744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fd0e3-af92-470d-970a-a2a6e4c04218 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.134795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 792/1594] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.134795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fd0e3-af92-470d-970a-a2a6e4c04218 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.142642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 803/1595] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:21.143487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d12b8-4dd3-402f-ac19-d254e30b6509 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.145861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fd0e3-af92-470d-970a-a2a6e4c04218 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.153404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a337d-8e52-41c5-b80e-177610ac0b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.154214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a337d-8e52-41c5-b80e-177610ac0b6d admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.154527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a337d-8e52-41c5-b80e-177610ac0b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.154647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d12b8-4dd3-402f-ac19-d254e30b6509 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.154820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a337d-8e52-41c5-b80e-177610ac0b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.155547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a337d-8e52-41c5-b80e-177610ac0b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.157192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a337d-8e52-41c5-b80e-177610ac0b6d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.159632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf334b4-daa5-445a-a828-b1ea978882ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.160222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf334b4-daa5-445a-a828-b1ea978882ac admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.160510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf334b4-daa5-445a-a828-b1ea978882ac admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.160783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf334b4-daa5-445a-a828-b1ea978882ac admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.161644 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf334b4-daa5-445a-a828-b1ea978882ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.163239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf334b4-daa5-445a-a828-b1ea978882ac admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.174262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 793/1596] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.175315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a337d-8e52-41c5-b80e-177610ac0b6d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.175518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 804/1597] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.176618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf334b4-daa5-445a-a828-b1ea978882ac admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.181773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:21.187989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf334b4-daa5-445a-a828-b1ea978882ac admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.188172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a337d-8e52-41c5-b80e-177610ac0b6d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.194911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb32f-0ffa-43fa-9eff-e971bda6c6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.195087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc78ebb-7cab-4d0d-9d02-6393442d1ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.195813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc78ebb-7cab-4d0d-9d02-6393442d1ebb admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.195947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb32f-0ffa-43fa-9eff-e971bda6c6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.196181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb32f-0ffa-43fa-9eff-e971bda6c6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.196403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc78ebb-7cab-4d0d-9d02-6393442d1ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.196531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb32f-0ffa-43fa-9eff-e971bda6c6ce admin admin] PATH_INFO: `/v3/projects/a6fd13a3dd1d4e2b83cf6e44160adf35/users/8eedefd8d94047478663ceab8962d738/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.196854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc78ebb-7cab-4d0d-9d02-6393442d1ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.197681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc78ebb-7cab-4d0d-9d02-6393442d1ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.200347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc78ebb-7cab-4d0d-9d02-6393442d1ebb admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.210955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 805/1598] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.214100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc78ebb-7cab-4d0d-9d02-6393442d1ebb admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.216724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfb32f-0ffa-43fa-9eff-e971bda6c6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fd13a3dd1d4e2b83cf6e44160adf35, user_id=8eedefd8d94047478663ceab8962d738, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.219643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfb32f-0ffa-43fa-9eff-e971bda6c6ce admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.225754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc78ebb-7cab-4d0d-9d02-6393442d1ebb admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.230999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b725-a2cb-4039-9b99-950152f885a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.232171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b725-a2cb-4039-9b99-950152f885a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.232517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b725-a2cb-4039-9b99-950152f885a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.232700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480b725-a2cb-4039-9b99-950152f885a8 admin admin] PATH_INFO: `/v3/projects/07562543fa3e40d584b4d4525b112319/users/62e566aa74b045cb8cc6e8edf4a256e1/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.250310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b725-a2cb-4039-9b99-950152f885a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07562543fa3e40d584b4d4525b112319, user_id=62e566aa74b045cb8cc6e8edf4a256e1, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.253102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480b725-a2cb-4039-9b99-950152f885a8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.290767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 794/1599] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:21 2022] PUT /identity/v3/projects/a6fd13a3dd1d4e2b83cf6e44160adf35/users/8eedefd8d94047478663ceab8962d738/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:21.291603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb32f-0ffa-43fa-9eff-e971bda6c6ce admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.307820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 806/1600] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:21 2022] PUT /identity/v3/projects/07562543fa3e40d584b4d4525b112319/users/62e566aa74b045cb8cc6e8edf4a256e1/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:21.310389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b725-a2cb-4039-9b99-950152f885a8 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.321673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:21.353857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb32f-0ffa-43fa-9eff-e971bda6c6ce admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.375343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480b725-a2cb-4039-9b99-950152f885a8 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.379941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d9b61-4c5d-4a7c-a5c8-27c9986e9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.380867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d9b61-4c5d-4a7c-a5c8-27c9986e9a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.381160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d9b61-4c5d-4a7c-a5c8-27c9986e9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.381458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d9b61-4c5d-4a7c-a5c8-27c9986e9a00 admin admin] PATH_INFO: `/v3/projects/ab084e94980d4f1abb02c9bf6fc50b74/users/e4807ff3644549529906d7f2fa9d5cbe/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.397537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d9b61-4c5d-4a7c-a5c8-27c9986e9a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab084e94980d4f1abb02c9bf6fc50b74, user_id=e4807ff3644549529906d7f2fa9d5cbe, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.400332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d9b61-4c5d-4a7c-a5c8-27c9986e9a00 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.410986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd192-3c28-4edc-9596-81d579ff1d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.411797 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd192-3c28-4edc-9596-81d579ff1d37 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.412093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd192-3c28-4edc-9596-81d579ff1d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.418892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd192-3c28-4edc-9596-81d579ff1d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.418892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd192-3c28-4edc-9596-81d579ff1d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.418892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd192-3c28-4edc-9596-81d579ff1d37 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.425879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 795/1601] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.426785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd192-3c28-4edc-9596-81d579ff1d37 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.437381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd192-3c28-4edc-9596-81d579ff1d37 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.442308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd892e-a760-41aa-a6ce-2b15b5c0b436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.447773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd892e-a760-41aa-a6ce-2b15b5c0b436 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.447773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd892e-a760-41aa-a6ce-2b15b5c0b436 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.447773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd892e-a760-41aa-a6ce-2b15b5c0b436 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.447773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd892e-a760-41aa-a6ce-2b15b5c0b436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.447773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd892e-a760-41aa-a6ce-2b15b5c0b436 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.460679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 796/1602] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:21.460679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64334c-2d87-4e92-b8bb-b43ef766ff3a None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.460679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64334c-2d87-4e92-b8bb-b43ef766ff3a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.460679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64334c-2d87-4e92-b8bb-b43ef766ff3a None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.461240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 797/1603] 10.222.0.40 () {54 vars in 962 bytes} [Mon Jul 25 05:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:59:21.461935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64334c-2d87-4e92-b8bb-b43ef766ff3a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.474598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 807/1604] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:21 2022] PUT /identity/v3/projects/ab084e94980d4f1abb02c9bf6fc50b74/users/e4807ff3644549529906d7f2fa9d5cbe/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:21.476047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07562d-896f-4954-a999-a66274dec7a9 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.476372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07562d-896f-4954-a999-a66274dec7a9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.476657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07562d-896f-4954-a999-a66274dec7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.481723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64334c-2d87-4e92-b8bb-b43ef766ff3a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.494502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1e07562d-896f-4954-a999-a66274dec7a9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:21.499319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1e07562d-896f-4954-a999-a66274dec7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:21.546561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7f9b8-eab7-4fd9-85f1-a1e180894109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.549683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7f9b8-eab7-4fd9-85f1-a1e180894109 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.550245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7f9b8-eab7-4fd9-85f1-a1e180894109 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.558908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc7f9b8-eab7-4fd9-85f1-a1e180894109 admin admin] PATH_INFO: `/v3/projects/9abeac7667fc471e8dcd18f455bb328f/users/b41c315652484a45993448bfa19247e2/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.569058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7f9b8-eab7-4fd9-85f1-a1e180894109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9abeac7667fc471e8dcd18f455bb328f, user_id=b41c315652484a45993448bfa19247e2, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.571720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc7f9b8-eab7-4fd9-85f1-a1e180894109 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.572939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 808/1605] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:21.575003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c3b8-3d7b-4188-b0ad-83ac8fba8256 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.575308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c3b8-3d7b-4188-b0ad-83ac8fba8256 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.575588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279c3b8-3d7b-4188-b0ad-83ac8fba8256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.599620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-6279c3b8-3d7b-4188-b0ad-83ac8fba8256 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:21.600854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6279c3b8-3d7b-4188-b0ad-83ac8fba8256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:21.609463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 809/1606] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:21.610243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c3b8-3d7b-4188-b0ad-83ac8fba8256 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.631658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279c3b8-3d7b-4188-b0ad-83ac8fba8256 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.635686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 798/1607] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:21 2022] PUT /identity/v3/projects/9abeac7667fc471e8dcd18f455bb328f/users/b41c315652484a45993448bfa19247e2/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:21.637633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7f9b8-eab7-4fd9-85f1-a1e180894109 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.701941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:21.705536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:21.719925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b196197-efa3-4948-8cb8-be0189520a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.720861 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196197-efa3-4948-8cb8-be0189520a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.721144 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196197-efa3-4948-8cb8-be0189520a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.721416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b196197-efa3-4948-8cb8-be0189520a9e admin admin] PATH_INFO: `/v3/projects/1484746571a24f5db36ed027469f88c1/users/db1a33dc0f174cb1bdebf9e0c710509d/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.725778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc7f9b8-eab7-4fd9-85f1-a1e180894109 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:21.732149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8425289-f54b-487a-943f-cf9af5fb9aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:21.737851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8425289-f54b-487a-943f-cf9af5fb9aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.737851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8425289-f54b-487a-943f-cf9af5fb9aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.737851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8425289-f54b-487a-943f-cf9af5fb9aaf admin admin] PATH_INFO: `/v3/projects/1871c9af781743f5af1d92371a42ecf1/users/d485f9fc38a442dd9e1528024460fcca/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.739351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b196197-efa3-4948-8cb8-be0189520a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1484746571a24f5db36ed027469f88c1, user_id=db1a33dc0f174cb1bdebf9e0c710509d, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.742508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b196197-efa3-4948-8cb8-be0189520a9e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.750661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8425289-f54b-487a-943f-cf9af5fb9aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1871c9af781743f5af1d92371a42ecf1, user_id=d485f9fc38a442dd9e1528024460fcca, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:21.754241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8425289-f54b-487a-943f-cf9af5fb9aaf admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:21.822521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 810/1608] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:21 2022] PUT /identity/v3/projects/1484746571a24f5db36ed027469f88c1/users/db1a33dc0f174cb1bdebf9e0c710509d/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:21.824160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ede3e8-6130-4135-b71c-63f3cc024e78 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.824483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ede3e8-6130-4135-b71c-63f3cc024e78 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.824766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ede3e8-6130-4135-b71c-63f3cc024e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.841215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 799/1609] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:21 2022] PUT /identity/v3/projects/1871c9af781743f5af1d92371a42ecf1/users/d485f9fc38a442dd9e1528024460fcca/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:21.842556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da056c-73dc-49bf-81ce-56dc26efea41 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:21.842853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da056c-73dc-49bf-81ce-56dc26efea41 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:21.843134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da056c-73dc-49bf-81ce-56dc26efea41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:21.882772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-26ede3e8-6130-4135-b71c-63f3cc024e78 None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:21.883998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-26ede3e8-6130-4135-b71c-63f3cc024e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:21.885394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-00da056c-73dc-49bf-81ce-56dc26efea41 None None] MFA Rules not processed for user `e4807ff3644549529906d7f2fa9d5cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:21.886950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-00da056c-73dc-49bf-81ce-56dc26efea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:21.953527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 800/1610] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:21.954314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da056c-73dc-49bf-81ce-56dc26efea41 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:21.958239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 811/1611] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 25 05:59:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:21.959169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ede3e8-6130-4135-b71c-63f3cc024e78 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.034533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ede3e8-6130-4135-b71c-63f3cc024e78 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.038314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f9b40-b65d-4188-8303-5c16358126c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.039124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f9b40-b65d-4188-8303-5c16358126c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.039412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f9b40-b65d-4188-8303-5c16358126c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.039690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f9b40-b65d-4188-8303-5c16358126c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.046302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da056c-73dc-49bf-81ce-56dc26efea41 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.050642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492817b-b07f-48d7-99e1-bc50cc150a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.056141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492817b-b07f-48d7-99e1-bc50cc150a2d service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.056141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492817b-b07f-48d7-99e1-bc50cc150a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.056141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492817b-b07f-48d7-99e1-bc50cc150a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.125981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:22.128589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f9b40-b65d-4188-8303-5c16358126c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.130116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f9b40-b65d-4188-8303-5c16358126c3 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.143310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 812/1612] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:22.144852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcab43-5e68-48ba-95f4-2afcf884284b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.145420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcab43-5e68-48ba-95f4-2afcf884284b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.145808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dcab43-5e68-48ba-95f4-2afcf884284b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.156825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492817b-b07f-48d7-99e1-bc50cc150a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.160315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492817b-b07f-48d7-99e1-bc50cc150a2d service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.169967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 801/1613] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:22.171481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7179-ffcc-4856-9fc6-9dda2efd492e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.171907 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7179-ffcc-4856-9fc6-9dda2efd492e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.172249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7179-ffcc-4856-9fc6-9dda2efd492e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.182817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:22.186416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b0dcab43-5e68-48ba-95f4-2afcf884284b None None] MFA Rules not processed for user `b41c315652484a45993448bfa19247e2`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:22.188700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b0dcab43-5e68-48ba-95f4-2afcf884284b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:22.200584 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-18ce7179-ffcc-4856-9fc6-9dda2efd492e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:22.201829 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-18ce7179-ffcc-4856-9fc6-9dda2efd492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:22.211427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 802/1614] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:22.212644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868af3-9763-473e-bd09-fc0575c81a3e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.212842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868af3-9763-473e-bd09-fc0575c81a3e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.213122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9868af3-9763-473e-bd09-fc0575c81a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.235627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e9868af3-9763-473e-bd09-fc0575c81a3e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:22.237856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e9868af3-9763-473e-bd09-fc0575c81a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:22.259535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 803/1615] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:22.260669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9868af3-9763-473e-bd09-fc0575c81a3e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.275444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9868af3-9763-473e-bd09-fc0575c81a3e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.279030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 813/1616] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:22.283524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e67b6-3249-4cab-936e-51d73c85cc57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.284594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e67b6-3249-4cab-936e-51d73c85cc57 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.284979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e67b6-3249-4cab-936e-51d73c85cc57 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.285373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b0bad-b0e8-4dc2-b050-69c53a05cd5f None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.285581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e67b6-3249-4cab-936e-51d73c85cc57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.286151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b0bad-b0e8-4dc2-b050-69c53a05cd5f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.286595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b0bad-b0e8-4dc2-b050-69c53a05cd5f None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.288560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 814/1617] 10.222.0.40 () {54 vars in 962 bytes} [Mon Jul 25 05:59:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:59:22.290572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b0bad-b0e8-4dc2-b050-69c53a05cd5f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.301764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b0bad-b0e8-4dc2-b050-69c53a05cd5f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.306267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda260b-50fc-40c2-b1d3-875c40087ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.307250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda260b-50fc-40c2-b1d3-875c40087ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.307720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda260b-50fc-40c2-b1d3-875c40087ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.308335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda260b-50fc-40c2-b1d3-875c40087ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.321994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda260b-50fc-40c2-b1d3-875c40087ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.324790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda260b-50fc-40c2-b1d3-875c40087ea2 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.346950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 815/1618] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:22.358949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda260b-50fc-40c2-b1d3-875c40087ea2 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.381728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e67b6-3249-4cab-936e-51d73c85cc57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.381728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e67b6-3249-4cab-936e-51d73c85cc57 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.381976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda260b-50fc-40c2-b1d3-875c40087ea2 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.386691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e9417-538b-4555-8f0d-e89bd1b000d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.387573 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9417-538b-4555-8f0d-e89bd1b000d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.387907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9417-538b-4555-8f0d-e89bd1b000d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.388466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9417-538b-4555-8f0d-e89bd1b000d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.398114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 804/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:22.399307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e67b6-3249-4cab-936e-51d73c85cc57 service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.409710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e9417-538b-4555-8f0d-e89bd1b000d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.411179 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e9417-538b-4555-8f0d-e89bd1b000d9 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.412705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e67b6-3249-4cab-936e-51d73c85cc57 service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.418288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da9fd4-eb08-4873-b315-947d27d4500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.419121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da9fd4-eb08-4873-b315-947d27d4500a admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.419483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da9fd4-eb08-4873-b315-947d27d4500a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.419835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da9fd4-eb08-4873-b315-947d27d4500a admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.420788 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da9fd4-eb08-4873-b315-947d27d4500a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.423460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da9fd4-eb08-4873-b315-947d27d4500a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.436707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 816/1620] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:22.439103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb7bfd-6225-49f7-a5b7-752ad5d34f32 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.439103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb7bfd-6225-49f7-a5b7-752ad5d34f32 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.439103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb7bfd-6225-49f7-a5b7-752ad5d34f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.467092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-cffb7bfd-6225-49f7-a5b7-752ad5d34f32 None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:22.468491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cffb7bfd-6225-49f7-a5b7-752ad5d34f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:22.488126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 817/1621] 10.222.0.40 () {58 vars in 1059 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:22.490832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb7bfd-6225-49f7-a5b7-752ad5d34f32 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.502868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb7bfd-6225-49f7-a5b7-752ad5d34f32 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.510361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772e7fe-285e-460c-a938-5c7419bc0cd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.512339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772e7fe-285e-460c-a938-5c7419bc0cd2 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.512799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772e7fe-285e-460c-a938-5c7419bc0cd2 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.513488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8772e7fe-285e-460c-a938-5c7419bc0cd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.527833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772e7fe-285e-460c-a938-5c7419bc0cd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.530231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8772e7fe-285e-460c-a938-5c7419bc0cd2 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.544905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 805/1622] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:22.546846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 818/1623] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:22.556096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da9fd4-eb08-4873-b315-947d27d4500a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.763268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da9fd4-eb08-4873-b315-947d27d4500a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.775766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ee053-2376-43ab-a6d7-74622feeb777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.776879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ee053-2376-43ab-a6d7-74622feeb777 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.776879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ee053-2376-43ab-a6d7-74622feeb777 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.777756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ee053-2376-43ab-a6d7-74622feeb777 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.778487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ee053-2376-43ab-a6d7-74622feeb777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.780607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ee053-2376-43ab-a6d7-74622feeb777 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.894744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 806/1624] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/users => generated 295 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:22.896761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772e7fe-285e-460c-a938-5c7419bc0cd2 service nova] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:22.919445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8772e7fe-285e-460c-a938-5c7419bc0cd2 service nova] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:22.931157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b34fa-77ad-4904-bd19-f6868cf86342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:22.932157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b34fa-77ad-4904-bd19-f6868cf86342 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.932686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b34fa-77ad-4904-bd19-f6868cf86342 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.933889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ad8db-7cd3-4326-a741-6ad28a87b139 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.934225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b34fa-77ad-4904-bd19-f6868cf86342 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.934456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ad8db-7cd3-4326-a741-6ad28a87b139 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.935412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b34fa-77ad-4904-bd19-f6868cf86342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:22.935617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ad8db-7cd3-4326-a741-6ad28a87b139 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.937068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 807/1625] 10.222.0.40 () {54 vars in 912 bytes} [Mon Jul 25 05:59:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:59:22.941851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b34fa-77ad-4904-bd19-f6868cf86342 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:22.945218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ce4a6-dd6a-4be3-a7e8-dd70229be1dc None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:22.945218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ce4a6-dd6a-4be3-a7e8-dd70229be1dc None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:22.945218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ce4a6-dd6a-4be3-a7e8-dd70229be1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:22.967244 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-738ce4a6-dd6a-4be3-a7e8-dd70229be1dc None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:22.968719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-738ce4a6-dd6a-4be3-a7e8-dd70229be1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:22.983041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 819/1626] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:22.986066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b34fa-77ad-4904-bd19-f6868cf86342 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.009876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b34fa-77ad-4904-bd19-f6868cf86342 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.025005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d48906-808d-4560-bcf2-a5c58e18781a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.034527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d48906-808d-4560-bcf2-a5c58e18781a admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.035297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d48906-808d-4560-bcf2-a5c58e18781a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.040286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d48906-808d-4560-bcf2-a5c58e18781a admin admin] PATH_INFO: `/v3/projects/8779958ba34f4c50a5c40935ac99ea6f/users/e5e402a2ca3642c3b4c2812c87b5d36a/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.063799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d48906-808d-4560-bcf2-a5c58e18781a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8779958ba34f4c50a5c40935ac99ea6f, user_id=e5e402a2ca3642c3b4c2812c87b5d36a, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.068616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:23.069023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d48906-808d-4560-bcf2-a5c58e18781a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.097780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 808/1627] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 05:59:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:23.099688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635c3a-3557-4383-859f-c34c3fb7ac5c None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.099799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635c3a-3557-4383-859f-c34c3fb7ac5c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.100013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76635c3a-3557-4383-859f-c34c3fb7ac5c None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.101439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 809/1628] 10.222.0.40 () {54 vars in 912 bytes} [Mon Jul 25 05:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:59:23.106649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76635c3a-3557-4383-859f-c34c3fb7ac5c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.158139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 820/1629] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:22 2022] PUT /identity/v3/projects/8779958ba34f4c50a5c40935ac99ea6f/users/e5e402a2ca3642c3b4c2812c87b5d36a/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:23.159567 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fc88a-7a6b-474c-8063-bad1cf510e59 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.160290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fc88a-7a6b-474c-8063-bad1cf510e59 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.160904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fc88a-7a6b-474c-8063-bad1cf510e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.181489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fc7fc88a-7a6b-474c-8063-bad1cf510e59 None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:23.182792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fc7fc88a-7a6b-474c-8063-bad1cf510e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:23.200792 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76635c3a-3557-4383-859f-c34c3fb7ac5c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.256860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:23.278399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 821/1630] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 05:59:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:23.279406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fc88a-7a6b-474c-8063-bad1cf510e59 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.309476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31027-a47b-44eb-bf7e-c307124af5a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.311090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31027-a47b-44eb-bf7e-c307124af5a2 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.311776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31027-a47b-44eb-bf7e-c307124af5a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.312172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e31027-a47b-44eb-bf7e-c307124af5a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.330781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e31027-a47b-44eb-bf7e-c307124af5a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.332990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e31027-a47b-44eb-bf7e-c307124af5a2 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.346942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 810/1631] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:23.347639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31027-a47b-44eb-bf7e-c307124af5a2 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.359343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e31027-a47b-44eb-bf7e-c307124af5a2 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.363607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa46201-b405-4265-9ade-99b03d1ff547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.365326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa46201-b405-4265-9ade-99b03d1ff547 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.365723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa46201-b405-4265-9ade-99b03d1ff547 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.366104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa46201-b405-4265-9ade-99b03d1ff547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.381506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa46201-b405-4265-9ade-99b03d1ff547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.384595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa46201-b405-4265-9ade-99b03d1ff547 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.387553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fc88a-7a6b-474c-8063-bad1cf510e59 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.392580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502acbf-965d-41da-b214-f443a5291457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.393398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502acbf-965d-41da-b214-f443a5291457 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.393829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502acbf-965d-41da-b214-f443a5291457 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.394250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502acbf-965d-41da-b214-f443a5291457 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.395093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502acbf-965d-41da-b214-f443a5291457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.396921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502acbf-965d-41da-b214-f443a5291457 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.409224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 811/1632] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:23.426416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 822/1633] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:23.430327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa46201-b405-4265-9ade-99b03d1ff547 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.455260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa46201-b405-4265-9ade-99b03d1ff547 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.461069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824140c-cc5e-4f71-886f-db16a3cda9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.462041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824140c-cc5e-4f71-886f-db16a3cda9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.462041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824140c-cc5e-4f71-886f-db16a3cda9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.462041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824140c-cc5e-4f71-886f-db16a3cda9d0 admin admin] PATH_INFO: `/v3/projects/8779958ba34f4c50a5c40935ac99ea6f/users/e5e402a2ca3642c3b4c2812c87b5d36a/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.469752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824140c-cc5e-4f71-886f-db16a3cda9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8779958ba34f4c50a5c40935ac99ea6f, user_id=e5e402a2ca3642c3b4c2812c87b5d36a, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:23.475766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824140c-cc5e-4f71-886f-db16a3cda9d0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:23.500238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 812/1634] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:23 2022] PUT /identity/v3/projects/8779958ba34f4c50a5c40935ac99ea6f/users/e5e402a2ca3642c3b4c2812c87b5d36a/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:23.510103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1630b-6bcc-413a-8aee-9a113a1e794f None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.512584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1630b-6bcc-413a-8aee-9a113a1e794f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.512584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc1630b-6bcc-413a-8aee-9a113a1e794f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.601117 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9cc1630b-6bcc-413a-8aee-9a113a1e794f None None] MFA Rules not processed for user `e5e402a2ca3642c3b4c2812c87b5d36a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:23.602599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9cc1630b-6bcc-413a-8aee-9a113a1e794f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:23.739539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824140c-cc5e-4f71-886f-db16a3cda9d0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.811565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 823/1635] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:23.818265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1630b-6bcc-413a-8aee-9a113a1e794f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:23.911460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824140c-cc5e-4f71-886f-db16a3cda9d0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:23.927307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:23.928613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:23.929105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:23.929574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] PATH_INFO: `/v3/users/c65f4282dd1e4f028d535b49febe12ee` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:23.935021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65f4282dd1e4f028d535b49febe12ee)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:24.165902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: WARNING py.warnings [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:59:24.165902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: warnings.warn(msg) Jul 25 05:59:24.165902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: Jul 25 05:59:24.165902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:24.168334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc1630b-6bcc-413a-8aee-9a113a1e794f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:24.168334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5823e4e-ed68-4ff7-ba0a-a47e61de2baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:24.168334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5823e4e-ed68-4ff7-ba0a-a47e61de2baa service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:24.168334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5823e4e-ed68-4ff7-ba0a-a47e61de2baa service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:24.171234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5823e4e-ed68-4ff7-ba0a-a47e61de2baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:24.171234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5823e4e-ed68-4ff7-ba0a-a47e61de2baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:24.171234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5823e4e-ed68-4ff7-ba0a-a47e61de2baa service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:24.171234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 824/1636] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:24.174188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:24.368574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5823e4e-ed68-4ff7-ba0a-a47e61de2baa service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:24.395545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5823e4e-ed68-4ff7-ba0a-a47e61de2baa service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:24.409765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c20e39-f2c2-41c4-8f15-77ea0dfbd2fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:24.410446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65f4282dd1e4f028d535b49febe12ee'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:24.416397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c20e39-f2c2-41c4-8f15-77ea0dfbd2fe service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:24.416819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c20e39-f2c2-41c4-8f15-77ea0dfbd2fe service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:24.417742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c20e39-f2c2-41c4-8f15-77ea0dfbd2fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:24.462761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:24.514593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65f4282dd1e4f028d535b49febe12ee'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:24.515219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65f4282dd1e4f028d535b49febe12ee'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:24.536607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65f4282dd1e4f028d535b49febe12ee'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:24.537946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65f4282dd1e4f028d535b49febe12ee'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:24.561168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c20e39-f2c2-41c4-8f15-77ea0dfbd2fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:24.566035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c20e39-f2c2-41c4-8f15-77ea0dfbd2fe service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:24.592336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 825/1637] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:24.602575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 813/1638] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:23 2022] DELETE /identity/v3/users/c65f4282dd1e4f028d535b49febe12ee => generated 0 bytes in 864 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:24.607013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288826a-1c98-4fd0-b76e-b61b3b6e1f16 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:24.607333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288826a-1c98-4fd0-b76e-b61b3b6e1f16 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:24.607598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288826a-1c98-4fd0-b76e-b61b3b6e1f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:24.639440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b288826a-1c98-4fd0-b76e-b61b3b6e1f16 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:24.640724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b288826a-1c98-4fd0-b76e-b61b3b6e1f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:24.752740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 826/1639] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:24.770913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:24.770913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:24.836988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd50aab-e4c5-4824-98c0-4854102d283a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:24.845312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b54693-e77b-4e5f-b117-c82f495491b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:24.845312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b54693-e77b-4e5f-b117-c82f495491b5 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:24.845312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b54693-e77b-4e5f-b117-c82f495491b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:24.845312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b54693-e77b-4e5f-b117-c82f495491b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:24.853225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b54693-e77b-4e5f-b117-c82f495491b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:24.855468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b54693-e77b-4e5f-b117-c82f495491b5 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:24.866141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 814/1640] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:26.736330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288826a-1c98-4fd0-b76e-b61b3b6e1f16 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:26.741243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b288826a-1c98-4fd0-b76e-b61b3b6e1f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:26.766953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288826a-1c98-4fd0-b76e-b61b3b6e1f16 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:26.779102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:26.780266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:26.780870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:26.781306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] PATH_INFO: `/v3/projects/e5643d28676f47869722468e73718165` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:26.784987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5643d28676f47869722468e73718165)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:26.787406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: WARNING py.warnings [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 25 05:59:26.787406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: warnings.warn(msg) Jul 25 05:59:26.787406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: Jul 25 05:59:26.790356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:27.012757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5643d28676f47869722468e73718165', 'request_id': 'req-0168f020-5941-4844-8046-71775a40ab3e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:27.134915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5643d28676f47869722468e73718165', 'request_id': 'req-0168f020-5941-4844-8046-71775a40ab3e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:27.154600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5643d28676f47869722468e73718165', 'request_id': 'req-0168f020-5941-4844-8046-71775a40ab3e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:27.213605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] The token cache is being invalidate because project e5643d28676f47869722468e73718165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:27.213758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:27.215265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0168f020-5941-4844-8046-71775a40ab3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:27.232176 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 827/1641] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:26 2022] DELETE /identity/v3/projects/e5643d28676f47869722468e73718165 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:27.233768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04ce2-ce43-43b6-bc84-2cebd2b29659 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:27.234715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04ce2-ce43-43b6-bc84-2cebd2b29659 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:27.235124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad04ce2-ce43-43b6-bc84-2cebd2b29659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:27.318438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8ad04ce2-ce43-43b6-bc84-2cebd2b29659 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:27.319666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8ad04ce2-ce43-43b6-bc84-2cebd2b29659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:27.505049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 815/1642] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:27.513467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de8a3-9c0b-43eb-985d-fb3e3caff99d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:27.513743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de8a3-9c0b-43eb-985d-fb3e3caff99d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:27.514025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de8a3-9c0b-43eb-985d-fb3e3caff99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:27.562815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0d1de8a3-9c0b-43eb-985d-fb3e3caff99d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:27.571575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0d1de8a3-9c0b-43eb-985d-fb3e3caff99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:27.602260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad04ce2-ce43-43b6-bc84-2cebd2b29659 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:27.617112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 828/1643] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:27.622154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad04ce2-ce43-43b6-bc84-2cebd2b29659 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:27.647416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:27.648155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:27.649889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:27.650164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:27.650929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:27.651247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:27.672142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:27.716162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 816/1644] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 05:59:27.720635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1de8a3-9c0b-43eb-985d-fb3e3caff99d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:27.736787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1de8a3-9c0b-43eb-985d-fb3e3caff99d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:27.751132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2989a44-032b-48a6-aba0-422716046ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:27.751837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2989a44-032b-48a6-aba0-422716046ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:27.752139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2989a44-032b-48a6-aba0-422716046ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:27.752481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2989a44-032b-48a6-aba0-422716046ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:27.753254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2989a44-032b-48a6-aba0-422716046ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:27.765837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2989a44-032b-48a6-aba0-422716046ce2 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:27.915291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 829/1645] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:27 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:27.916692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:27.933422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd8b32-d81e-425c-827e-a309ebd60eae admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:28.016807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:28.102343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423ba4c-4f89-472b-8851-b9b8868d3736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:28.103653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423ba4c-4f89-472b-8851-b9b8868d3736 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.103653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423ba4c-4f89-472b-8851-b9b8868d3736 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.103849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423ba4c-4f89-472b-8851-b9b8868d3736 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.105463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423ba4c-4f89-472b-8851-b9b8868d3736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:28.110832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423ba4c-4f89-472b-8851-b9b8868d3736 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:28.234075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 817/1646] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:27 2022] POST /identity/v3/users => generated 287 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:28.238459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2989a44-032b-48a6-aba0-422716046ce2 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:28.255809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2989a44-032b-48a6-aba0-422716046ce2 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:28.266680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbaeb7-7361-409f-97ad-cf55944c1035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:28.271604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbaeb7-7361-409f-97ad-cf55944c1035 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.271785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbaeb7-7361-409f-97ad-cf55944c1035 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.275685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbaeb7-7361-409f-97ad-cf55944c1035 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.275685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbaeb7-7361-409f-97ad-cf55944c1035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:28.278399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbaeb7-7361-409f-97ad-cf55944c1035 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:28.303861 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 830/1647] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:28.308204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423ba4c-4f89-472b-8851-b9b8868d3736 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:28.326079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423ba4c-4f89-472b-8851-b9b8868d3736 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:28.342710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd525a89-d747-44b1-b2cf-e0afbc677635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:28.344877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd525a89-d747-44b1-b2cf-e0afbc677635 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.345415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd525a89-d747-44b1-b2cf-e0afbc677635 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.346416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd525a89-d747-44b1-b2cf-e0afbc677635 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.350287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd525a89-d747-44b1-b2cf-e0afbc677635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:28.361551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd525a89-d747-44b1-b2cf-e0afbc677635 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:28.390743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 818/1648] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:28.395472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbaeb7-7361-409f-97ad-cf55944c1035 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:28.403014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:28.409659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbaeb7-7361-409f-97ad-cf55944c1035 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:28.419892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a263b-4f34-4ef0-b518-3e6e06acde6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:28.423142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a263b-4f34-4ef0-b518-3e6e06acde6a admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.423674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a263b-4f34-4ef0-b518-3e6e06acde6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.424166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a263b-4f34-4ef0-b518-3e6e06acde6a admin admin] PATH_INFO: `/v3/projects/f7b940a05c91473eb2f7cd30484de91f/users/5b1b05dfe33f41529e60b8602b38e14e/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.444380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a263b-4f34-4ef0-b518-3e6e06acde6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7b940a05c91473eb2f7cd30484de91f, user_id=5b1b05dfe33f41529e60b8602b38e14e, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:28.447344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a263b-4f34-4ef0-b518-3e6e06acde6a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:28.572177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 831/1649] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:28 2022] PUT /identity/v3/projects/f7b940a05c91473eb2f7cd30484de91f/users/5b1b05dfe33f41529e60b8602b38e14e/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:28.586512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc87f7e-4af8-4b29-8bcb-c9fceae9aaf1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:28.587195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc87f7e-4af8-4b29-8bcb-c9fceae9aaf1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:28.587672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc87f7e-4af8-4b29-8bcb-c9fceae9aaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:28.653004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9cc87f7e-4af8-4b29-8bcb-c9fceae9aaf1 None None] MFA Rules not processed for user `5b1b05dfe33f41529e60b8602b38e14e`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:28.656564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9cc87f7e-4af8-4b29-8bcb-c9fceae9aaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:28.915037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 819/1650] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:28.919487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a263b-4f34-4ef0-b518-3e6e06acde6a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.012354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a263b-4f34-4ef0-b518-3e6e06acde6a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:29.024306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8f6b4-a7be-43ac-bd91-881cfbc82054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:29.027216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8f6b4-a7be-43ac-bd91-881cfbc82054 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:29.027853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8f6b4-a7be-43ac-bd91-881cfbc82054 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:29.028397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8f6b4-a7be-43ac-bd91-881cfbc82054 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:29.029329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8f6b4-a7be-43ac-bd91-881cfbc82054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:29.034587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8f6b4-a7be-43ac-bd91-881cfbc82054 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:29.167055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc87f7e-4af8-4b29-8bcb-c9fceae9aaf1 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.173372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 832/1651] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:28 2022] POST /identity/v3/projects => generated 373 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:29.187794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc87f7e-4af8-4b29-8bcb-c9fceae9aaf1 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:29.318177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b5170-3d9e-4157-93d1-9e73b65473be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:29.318881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b5170-3d9e-4157-93d1-9e73b65473be admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:29.319240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b5170-3d9e-4157-93d1-9e73b65473be admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:29.319621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b5170-3d9e-4157-93d1-9e73b65473be admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:29.320427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b5170-3d9e-4157-93d1-9e73b65473be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:29.322567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b5170-3d9e-4157-93d1-9e73b65473be admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:29.434633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 820/1652] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:29 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:29.439050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8f6b4-a7be-43ac-bd91-881cfbc82054 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.460767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8f6b4-a7be-43ac-bd91-881cfbc82054 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:29.477561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e889439-9819-4ae1-9cd2-8450d5d25b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:29.478716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e889439-9819-4ae1-9cd2-8450d5d25b8a admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:29.479171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e889439-9819-4ae1-9cd2-8450d5d25b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:29.479504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e889439-9819-4ae1-9cd2-8450d5d25b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:29.480481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e889439-9819-4ae1-9cd2-8450d5d25b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:29.482135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e889439-9819-4ae1-9cd2-8450d5d25b8a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:29.508681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 833/1653] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:29.514131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b5170-3d9e-4157-93d1-9e73b65473be admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.534168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b5170-3d9e-4157-93d1-9e73b65473be admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:29.544914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f1de3-6069-4b4a-8215-6593607cd009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:29.545863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f1de3-6069-4b4a-8215-6593607cd009 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:29.546178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f1de3-6069-4b4a-8215-6593607cd009 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:29.546663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f1de3-6069-4b4a-8215-6593607cd009 admin admin] PATH_INFO: `/v3/projects/86b38e015e404f71901d84f1bd38e0bd/users/e41c7362b03c4a9ca3c5e117a83adec4/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:29.592622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f1de3-6069-4b4a-8215-6593607cd009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b38e015e404f71901d84f1bd38e0bd, user_id=e41c7362b03c4a9ca3c5e117a83adec4, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:29.609082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f1de3-6069-4b4a-8215-6593607cd009 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:29.748535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e889439-9819-4ae1-9cd2-8450d5d25b8a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:29.763988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 821/1654] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:29 2022] PUT /identity/v3/projects/86b38e015e404f71901d84f1bd38e0bd/users/e41c7362b03c4a9ca3c5e117a83adec4/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:29.807142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e889439-9819-4ae1-9cd2-8450d5d25b8a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:30.000272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b3602-5646-4f62-acbf-6bc5087c1a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:30.001000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b3602-5646-4f62-acbf-6bc5087c1a67 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:30.001315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b3602-5646-4f62-acbf-6bc5087c1a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:30.004072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b3602-5646-4f62-acbf-6bc5087c1a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:30.004072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b3602-5646-4f62-acbf-6bc5087c1a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:30.014004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b3602-5646-4f62-acbf-6bc5087c1a67 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:30.076673 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 834/1655] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:29 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:30.082152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f1de3-6069-4b4a-8215-6593607cd009 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:30.106382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f1de3-6069-4b4a-8215-6593607cd009 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:30.127016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:30.128209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:30.128510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:30.128795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] PATH_INFO: `/v3/projects/86b38e015e404f71901d84f1bd38e0bd/users/e41c7362b03c4a9ca3c5e117a83adec4/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:30.131101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86b38e015e404f71901d84f1bd38e0bd, user_id=e41c7362b03c4a9ca3c5e117a83adec4, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:30.153752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:30.197167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:30.241226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eeeff-5aa7-43fe-89e7-7dd637c0918d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:30.241807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eeeff-5aa7-43fe-89e7-7dd637c0918d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:30.243131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 822/1656] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:30 2022] PUT /identity/v3/projects/86b38e015e404f71901d84f1bd38e0bd/users/e41c7362b03c4a9ca3c5e117a83adec4/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:30.243715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11eeeff-5aa7-43fe-89e7-7dd637c0918d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:30.325007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f11eeeff-5aa7-43fe-89e7-7dd637c0918d None None] MFA Rules not processed for user `e41c7362b03c4a9ca3c5e117a83adec4`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:30.326584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f11eeeff-5aa7-43fe-89e7-7dd637c0918d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:30.457115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:30.525447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 835/1657] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:30.555098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:30.560957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:30.735197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887bc00-ea5a-439e-b344-5e8efabb555d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:30.744926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e10ac-aded-492f-8c57-0f9d690de5ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:30.746186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e10ac-aded-492f-8c57-0f9d690de5ec service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:30.746572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e10ac-aded-492f-8c57-0f9d690de5ec service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:30.747917 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e10ac-aded-492f-8c57-0f9d690de5ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:30.772271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e10ac-aded-492f-8c57-0f9d690de5ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:30.773756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e10ac-aded-492f-8c57-0f9d690de5ec service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:30.808059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 823/1658] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:31.084037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11eeeff-5aa7-43fe-89e7-7dd637c0918d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:31.086530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f11eeeff-5aa7-43fe-89e7-7dd637c0918d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:31.118077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11eeeff-5aa7-43fe-89e7-7dd637c0918d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:31.132667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:31.133691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:31.134410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:31.134882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:31.241468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:31.244206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:31.260783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 836/1659] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:33.640894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f981e6c-4219-4be6-95fe-a04b65da6e18 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:33.641802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f981e6c-4219-4be6-95fe-a04b65da6e18 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:33.642349 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f981e6c-4219-4be6-95fe-a04b65da6e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:33.684176 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-0f981e6c-4219-4be6-95fe-a04b65da6e18 None None] MFA Rules not processed for user `a7224418acee432182bf3195bfc8fd99`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:33.686328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0f981e6c-4219-4be6-95fe-a04b65da6e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:33.831154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 824/1660] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:33.845214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:33.846683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:33.992479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d433aae-067c-4db5-a90f-4aa5cf302a52 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:34.013707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c6b4f-6642-4890-98b5-fa5329955319 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:34.014628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c6b4f-6642-4890-98b5-fa5329955319 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:34.015120 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c6b4f-6642-4890-98b5-fa5329955319 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:34.015531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c6b4f-6642-4890-98b5-fa5329955319 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:34.031754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c6b4f-6642-4890-98b5-fa5329955319 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:34.038098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c6b4f-6642-4890-98b5-fa5329955319 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:34.062421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 837/1661] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:36.953482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998130f5-ec1b-4833-924e-d16657bba14a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:36.953973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998130f5-ec1b-4833-924e-d16657bba14a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:36.954099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998130f5-ec1b-4833-924e-d16657bba14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.052290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-998130f5-ec1b-4833-924e-d16657bba14a None None] MFA Rules not processed for user `db1a33dc0f174cb1bdebf9e0c710509d`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:37.056202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-998130f5-ec1b-4833-924e-d16657bba14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:37.226620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 825/1662] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:37.227294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdabec7-13e5-4742-8c1d-c28f1235b19c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.227801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdabec7-13e5-4742-8c1d-c28f1235b19c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.227889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdabec7-13e5-4742-8c1d-c28f1235b19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.256250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-cfdabec7-13e5-4742-8c1d-c28f1235b19c None None] MFA Rules not processed for user `db1a33dc0f174cb1bdebf9e0c710509d`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:37.257403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cfdabec7-13e5-4742-8c1d-c28f1235b19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:37.295071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 838/1663] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:37.310161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998130f5-ec1b-4833-924e-d16657bba14a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.312336 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-998130f5-ec1b-4833-924e-d16657bba14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:37.367391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998130f5-ec1b-4833-924e-d16657bba14a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:37.378792 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a688f9-4f73-46aa-a861-a815ad683324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:37.379815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a688f9-4f73-46aa-a861-a815ad683324 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:37.386671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a688f9-4f73-46aa-a861-a815ad683324 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:37.387215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a688f9-4f73-46aa-a861-a815ad683324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:37.399046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a688f9-4f73-46aa-a861-a815ad683324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:37.402368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a688f9-4f73-46aa-a861-a815ad683324 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:37.410146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:37.417876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 826/1664] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:37.770893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdabec7-13e5-4742-8c1d-c28f1235b19c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:37.779756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cfdabec7-13e5-4742-8c1d-c28f1235b19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:37.946404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:37.991211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdabec7-13e5-4742-8c1d-c28f1235b19c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:38.003655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:38.005385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.009892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.010925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] PATH_INFO: `/v3/users/e4807ff3644549529906d7f2fa9d5cbe` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.013859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4807ff3644549529906d7f2fa9d5cbe)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:38.028493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:38.244980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4807ff3644549529906d7f2fa9d5cbe'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.358765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4807ff3644549529906d7f2fa9d5cbe'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.359634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4807ff3644549529906d7f2fa9d5cbe'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.368724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4807ff3644549529906d7f2fa9d5cbe'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.371568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4807ff3644549529906d7f2fa9d5cbe'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:38.398188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 839/1665] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:37 2022] DELETE /identity/v3/users/e4807ff3644549529906d7f2fa9d5cbe => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:38.403680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf85ff-9a31-4351-ae9e-5763e8578048 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.404622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf85ff-9a31-4351-ae9e-5763e8578048 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.405831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf85ff-9a31-4351-ae9e-5763e8578048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.445097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-59cf85ff-9a31-4351-ae9e-5763e8578048 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:38.453866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-59cf85ff-9a31-4351-ae9e-5763e8578048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.631613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 827/1666] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:38.694259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:38.695777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.851505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18adbaf5-582f-40d9-909a-11d74a1c9dd5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:38.859104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ca78c-5284-4bdb-84ce-e51e66dc3ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:38.862339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca78c-5284-4bdb-84ce-e51e66dc3ccc service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.862339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca78c-5284-4bdb-84ce-e51e66dc3ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.862339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca78c-5284-4bdb-84ce-e51e66dc3ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.892554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18d0b2-7e84-40e4-bb59-0e18fd7c4234 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.892856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18d0b2-7e84-40e4-bb59-0e18fd7c4234 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.893143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18d0b2-7e84-40e4-bb59-0e18fd7c4234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:38.901767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ca78c-5284-4bdb-84ce-e51e66dc3ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:38.903416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ca78c-5284-4bdb-84ce-e51e66dc3ccc service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:38.917962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 840/1667] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:38.952163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4b18d0b2-7e84-40e4-bb59-0e18fd7c4234 None None] MFA Rules not processed for user `8eedefd8d94047478663ceab8962d738`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:38.952739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4b18d0b2-7e84-40e4-bb59-0e18fd7c4234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:38.994903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a474b-10b3-4e45-9d60-2185ea1fee85 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:38.997357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a474b-10b3-4e45-9d60-2185ea1fee85 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:38.997706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a474b-10b3-4e45-9d60-2185ea1fee85 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.001800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 841/1668] 10.222.0.40 () {54 vars in 912 bytes} [Mon Jul 25 05:59:38 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 25 05:59:39.008295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab384020-3788-40ef-9a52-09d16997117b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.008742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab384020-3788-40ef-9a52-09d16997117b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.008964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab384020-3788-40ef-9a52-09d16997117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.092912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ab384020-3788-40ef-9a52-09d16997117b None None] MFA Rules not processed for user `b1638c28af8240439d061a3ae0ddc38d`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:39.097450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ab384020-3788-40ef-9a52-09d16997117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:39.140541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 842/1669] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 05:59:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:39.148175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 828/1670] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:39.165256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab384020-3788-40ef-9a52-09d16997117b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:39.168834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18d0b2-7e84-40e4-bb59-0e18fd7c4234 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:39.170791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4b18d0b2-7e84-40e4-bb59-0e18fd7c4234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:39.171444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ab384020-3788-40ef-9a52-09d16997117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:39.194663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab384020-3788-40ef-9a52-09d16997117b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:39.204962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4dd03-56ef-4346-8528-76a1890f02b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:39.204962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4dd03-56ef-4346-8528-76a1890f02b6 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.204962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4dd03-56ef-4346-8528-76a1890f02b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.204962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4dd03-56ef-4346-8528-76a1890f02b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.231326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4dd03-56ef-4346-8528-76a1890f02b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:39.232612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4dd03-56ef-4346-8528-76a1890f02b6 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:39.266033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 843/1671] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:39.383253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18d0b2-7e84-40e4-bb59-0e18fd7c4234 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:39.405115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024430e-cc48-4f30-8b4d-fd2726124f9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:39.406297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024430e-cc48-4f30-8b4d-fd2726124f9e service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:39.406574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024430e-cc48-4f30-8b4d-fd2726124f9e service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:39.407733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024430e-cc48-4f30-8b4d-fd2726124f9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:39.426424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024430e-cc48-4f30-8b4d-fd2726124f9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:39.430846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9024430e-cc48-4f30-8b4d-fd2726124f9e service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:39.472207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 829/1672] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:40.397896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a217-b57d-422a-8a7e-ab6e9b1e640e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.398351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a217-b57d-422a-8a7e-ab6e9b1e640e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.398463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810a217-b57d-422a-8a7e-ab6e9b1e640e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.492051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1810a217-b57d-422a-8a7e-ab6e9b1e640e None None] MFA Rules not processed for user `d485f9fc38a442dd9e1528024460fcca`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:40.493669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1810a217-b57d-422a-8a7e-ab6e9b1e640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:40.567966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:40.655815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 844/1673] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:40.672343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024430e-cc48-4f30-8b4d-fd2726124f9e service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:40.698247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024430e-cc48-4f30-8b4d-fd2726124f9e service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:40.713928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce08308-73c7-467b-8c51-6731e4837a8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:40.719965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce08308-73c7-467b-8c51-6731e4837a8c service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.723985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce08308-73c7-467b-8c51-6731e4837a8c service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.723985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce08308-73c7-467b-8c51-6731e4837a8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.749454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce08308-73c7-467b-8c51-6731e4837a8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:40.753302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce08308-73c7-467b-8c51-6731e4837a8c service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:40.808304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 830/1674] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:40.841273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810a217-b57d-422a-8a7e-ab6e9b1e640e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:40.843028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1810a217-b57d-422a-8a7e-ab6e9b1e640e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:40.879979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810a217-b57d-422a-8a7e-ab6e9b1e640e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:40.914066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:40.915221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:40.920668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:40.921248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] PATH_INFO: `/v3/projects/ab084e94980d4f1abb02c9bf6fc50b74` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:40.925667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab084e94980d4f1abb02c9bf6fc50b74)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:40.931078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.022293 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35dc0d-d53f-4fea-8a8a-286eacf6887f None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.022825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35dc0d-d53f-4fea-8a8a-286eacf6887f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.023209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35dc0d-d53f-4fea-8a8a-286eacf6887f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.030273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:41.068753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2f35dc0d-d53f-4fea-8a8a-286eacf6887f None None] MFA Rules not processed for user `62e566aa74b045cb8cc6e8edf4a256e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.084123 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2f35dc0d-d53f-4fea-8a8a-286eacf6887f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.118838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab084e94980d4f1abb02c9bf6fc50b74', 'request_id': 'req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.205785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 831/1675] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:41.209953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260b17-85a4-4436-a3a3-bd612c0b4213 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.210251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260b17-85a4-4436-a3a3-bd612c0b4213 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.210532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1260b17-85a4-4436-a3a3-bd612c0b4213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.256443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f1260b17-85a4-4436-a3a3-bd612c0b4213 None None] MFA Rules not processed for user `62e566aa74b045cb8cc6e8edf4a256e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.260530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f1260b17-85a4-4436-a3a3-bd612c0b4213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.272683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab084e94980d4f1abb02c9bf6fc50b74', 'request_id': 'req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.285521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab084e94980d4f1abb02c9bf6fc50b74', 'request_id': 'req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.304319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 832/1676] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:41.316135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1260b17-85a4-4436-a3a3-bd612c0b4213 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.319029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f1260b17-85a4-4436-a3a3-bd612c0b4213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.327954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] The token cache is being invalidate because project ab084e94980d4f1abb02c9bf6fc50b74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:41.331542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.332747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cae8c5a-5e34-4747-98d7-b8e2bfc388f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:41.339734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 845/1677] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:40 2022] DELETE /identity/v3/projects/ab084e94980d4f1abb02c9bf6fc50b74 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:41.346314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27c235-b3d2-41d6-990e-fb9ad7102288 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.346629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27c235-b3d2-41d6-990e-fb9ad7102288 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.352304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27c235-b3d2-41d6-990e-fb9ad7102288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.439114 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1d27c235-b3d2-41d6-990e-fb9ad7102288 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.443366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1d27c235-b3d2-41d6-990e-fb9ad7102288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.585320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 846/1678] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:41.587488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a96f-8b3e-42d8-9216-15b4e42a72ea None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.588516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a96f-8b3e-42d8-9216-15b4e42a72ea None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.597281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927a96f-8b3e-42d8-9216-15b4e42a72ea None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.597281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 847/1679] 10.222.0.40 () {54 vars in 961 bytes} [Mon Jul 25 05:59:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:59:41.601242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b2654-5c84-4aac-a4e0-fb697d1bcbcc None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.601537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b2654-5c84-4aac-a4e0-fb697d1bcbcc None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.601828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40b2654-5c84-4aac-a4e0-fb697d1bcbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.607655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1260b17-85a4-4436-a3a3-bd612c0b4213 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.617076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.618162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.620820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.620820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.622441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.622577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d40b2654-5c84-4aac-a4e0-fb697d1bcbcc None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.624380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d40b2654-5c84-4aac-a4e0-fb697d1bcbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.638516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 848/1680] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:41.640027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c89fc-833c-4a1e-b114-0d938e126688 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.640507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c89fc-833c-4a1e-b114-0d938e126688 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.640799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c89fc-833c-4a1e-b114-0d938e126688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.644190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.646168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.660450 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 833/1681] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:41.661761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.689254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b836e07-7b1d-43e4-ba17-47976fe4fb20 service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.695233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.696149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.696645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.697063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.699010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:41.699814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.702665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.710668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0c2c89fc-833c-4a1e-b114-0d938e126688 None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:41.710668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0c2c89fc-833c-4a1e-b114-0d938e126688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.727797 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 834/1682] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:41.730158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.792541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322fe40-e60b-4041-8276-0a1e23f81d3a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.799697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:41.802625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b49e8-1e4a-42cf-831c-6ba6809697c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.803365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b49e8-1e4a-42cf-831c-6ba6809697c9 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.803673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b49e8-1e4a-42cf-831c-6ba6809697c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.804002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b49e8-1e4a-42cf-831c-6ba6809697c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.804735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b49e8-1e4a-42cf-831c-6ba6809697c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.808521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b49e8-1e4a-42cf-831c-6ba6809697c9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.842685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 849/1683] 10.222.0.40 () {58 vars in 1058 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:41.856998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c89fc-833c-4a1e-b114-0d938e126688 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.874594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c89fc-833c-4a1e-b114-0d938e126688 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:41.887215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:41.887902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:41.888287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:41.888688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:41.891440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:41.910599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:41.912761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:41.937393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 835/1684] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/projects => generated 365 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 05:59:41.950354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b49e8-1e4a-42cf-831c-6ba6809697c9 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:41.956256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 850/1685] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:42.053309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:42.085511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b49e8-1e4a-42cf-831c-6ba6809697c9 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.093962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb299-0404-49dc-9393-52270164fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.094764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb299-0404-49dc-9393-52270164fb03 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.095114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb299-0404-49dc-9393-52270164fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.102790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fb299-0404-49dc-9393-52270164fb03 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.102790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fb299-0404-49dc-9393-52270164fb03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.102790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fb299-0404-49dc-9393-52270164fb03 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.230408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 836/1686] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:41 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:42.232954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.255970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673273e1-b640-4add-aa66-ce2614c97f3b service placement] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.275158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12616ecf-f6dc-4cf1-9067-ae121cba8012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.277377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12616ecf-f6dc-4cf1-9067-ae121cba8012 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.277377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12616ecf-f6dc-4cf1-9067-ae121cba8012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.277662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12616ecf-f6dc-4cf1-9067-ae121cba8012 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.278292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12616ecf-f6dc-4cf1-9067-ae121cba8012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.279939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12616ecf-f6dc-4cf1-9067-ae121cba8012 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.316233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 851/1687] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:42.318341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb299-0404-49dc-9393-52270164fb03 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.361833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fb299-0404-49dc-9393-52270164fb03 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.391754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.392494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.392785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.393065 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.393818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.405166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.440788 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 837/1688] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:42.444913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12616ecf-f6dc-4cf1-9067-ae121cba8012 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.460092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12616ecf-f6dc-4cf1-9067-ae121cba8012 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.468822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f510c5d-72d7-4ed3-89ab-e00332c9deee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.469674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510c5d-72d7-4ed3-89ab-e00332c9deee admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.470244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510c5d-72d7-4ed3-89ab-e00332c9deee admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.470534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f510c5d-72d7-4ed3-89ab-e00332c9deee admin admin] PATH_INFO: `/v3/projects/78324bbf84ef41a0867446a5f8e6925a/users/6c6486c089e14df29cf450f033b9e317/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.498698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f510c5d-72d7-4ed3-89ab-e00332c9deee admin admin] RBAC: Authorizing `identity:create_grant(project_id=78324bbf84ef41a0867446a5f8e6925a, user_id=6c6486c089e14df29cf450f033b9e317, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.501778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f510c5d-72d7-4ed3-89ab-e00332c9deee admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.583737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:42.583737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:42.599700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf2dfd-8a40-4c94-9e2a-55cce0f4eb8d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:42.621296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:42.621296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.621296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.621296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] PATH_INFO: `/v3/users/b41c315652484a45993448bfa19247e2` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.658454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41c315652484a45993448bfa19247e2)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:42.661171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:42.661690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 852/1689] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:42 2022] PUT /identity/v3/projects/78324bbf84ef41a0867446a5f8e6925a/users/6c6486c089e14df29cf450f033b9e317/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:42.663481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba26cb-42c8-40bf-b246-9ff2c8783d74 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:42.663980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba26cb-42c8-40bf-b246-9ff2c8783d74 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:42.664515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba26cb-42c8-40bf-b246-9ff2c8783d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:42.738562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-31ba26cb-42c8-40bf-b246-9ff2c8783d74 None None] MFA Rules not processed for user `6c6486c089e14df29cf450f033b9e317`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:42.740309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-31ba26cb-42c8-40bf-b246-9ff2c8783d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:42.988645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41c315652484a45993448bfa19247e2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:43.043170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 853/1690] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:43.048024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba26cb-42c8-40bf-b246-9ff2c8783d74 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:43.092035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41c315652484a45993448bfa19247e2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:43.092759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41c315652484a45993448bfa19247e2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:43.105664 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41c315652484a45993448bfa19247e2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:43.106494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1de38ea1-8cf7-42e7-b69e-05c29da57f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41c315652484a45993448bfa19247e2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:43.173626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 838/1691] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:42 2022] DELETE /identity/v3/users/b41c315652484a45993448bfa19247e2 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:43.178794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e266eb5-5141-4358-ab4e-d5dbe0383f03 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:43.179135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e266eb5-5141-4358-ab4e-d5dbe0383f03 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:43.181969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e266eb5-5141-4358-ab4e-d5dbe0383f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:43.212229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba26cb-42c8-40bf-b246-9ff2c8783d74 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:43.237429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3e266eb5-5141-4358-ab4e-d5dbe0383f03 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:43.240639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3e266eb5-5141-4358-ab4e-d5dbe0383f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:43.256828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:43.397946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:43.402644 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:43.403050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:43.403375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:43.410909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:43.414685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:43.425218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 839/1692] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:43.426432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9ba51-30af-4ba4-83e9-db8e18c62e05 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:43.426751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9ba51-30af-4ba4-83e9-db8e18c62e05 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:43.426997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9ba51-30af-4ba4-83e9-db8e18c62e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:43.461896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-43a9ba51-30af-4ba4-83e9-db8e18c62e05 None None] MFA Rules not processed for user `a7224418acee432182bf3195bfc8fd99`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:43.464735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-43a9ba51-30af-4ba4-83e9-db8e18c62e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:43.506004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:43.594797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 854/1693] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:43 2022] POST /identity/v3/projects => generated 365 bytes in 547 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:43.595868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:43.603063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:43.633029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 840/1694] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:43.643412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9ba51-30af-4ba4-83e9-db8e18c62e05 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:43.650104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-43a9ba51-30af-4ba4-83e9-db8e18c62e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:43.690942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9ba51-30af-4ba4-83e9-db8e18c62e05 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:43.868832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ecdd9-3873-41cb-a4a5-abce44727d92 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:43.883297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff6d2c-d7fa-4a63-9c20-cce36722e8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:43.884026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff6d2c-d7fa-4a63-9c20-cce36722e8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:43.887630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff6d2c-d7fa-4a63-9c20-cce36722e8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:43.890088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff6d2c-d7fa-4a63-9c20-cce36722e8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:43.912687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac8887-725d-47bb-9350-533924fd0611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:43.915470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac8887-725d-47bb-9350-533924fd0611 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:43.915784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac8887-725d-47bb-9350-533924fd0611 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:43.918462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac8887-725d-47bb-9350-533924fd0611 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:43.918846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac8887-725d-47bb-9350-533924fd0611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:43.924192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac8887-725d-47bb-9350-533924fd0611 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:44.046954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff6d2c-d7fa-4a63-9c20-cce36722e8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:44.048938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff6d2c-d7fa-4a63-9c20-cce36722e8b4 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:44.059750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 841/1695] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:43 2022] POST /identity/v3/users => generated 289 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:44.061307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01c69e-89bb-4170-9c72-2471d1fa8eae None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:44.061604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01c69e-89bb-4170-9c72-2471d1fa8eae None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:44.062561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b01c69e-89bb-4170-9c72-2471d1fa8eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:44.088684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 855/1696] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:44.088684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff6d2c-d7fa-4a63-9c20-cce36722e8b4 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:44.108140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1b01c69e-89bb-4170-9c72-2471d1fa8eae None None] MFA Rules not processed for user `62e566aa74b045cb8cc6e8edf4a256e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:44.109424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1b01c69e-89bb-4170-9c72-2471d1fa8eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:44.120739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff6d2c-d7fa-4a63-9c20-cce36722e8b4 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:44.138297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13f934-b330-40dc-b20c-bd6bb077d4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:44.139017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f934-b330-40dc-b20c-bd6bb077d4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:44.143832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f934-b330-40dc-b20c-bd6bb077d4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:44.147279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13f934-b330-40dc-b20c-bd6bb077d4c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:44.149039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13f934-b330-40dc-b20c-bd6bb077d4c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:44.158418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13f934-b330-40dc-b20c-bd6bb077d4c0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:44.200578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 856/1697] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:44.206353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13f934-b330-40dc-b20c-bd6bb077d4c0 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:44.244822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13f934-b330-40dc-b20c-bd6bb077d4c0 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:44.256685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cf6de-c1f4-4838-a4d6-a18ef735ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:44.258844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cf6de-c1f4-4838-a4d6-a18ef735ed9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:44.259147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cf6de-c1f4-4838-a4d6-a18ef735ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:44.261870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90cf6de-c1f4-4838-a4d6-a18ef735ed9f admin admin] PATH_INFO: `/v3/projects/ea748f3b1bfe4433a4e4c1cde60ea741/users/1538406d73824e53ae6b8ec9a4a092af/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:44.339812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90cf6de-c1f4-4838-a4d6-a18ef735ed9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea748f3b1bfe4433a4e4c1cde60ea741, user_id=1538406d73824e53ae6b8ec9a4a092af, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:44.344371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90cf6de-c1f4-4838-a4d6-a18ef735ed9f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:44.363230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 842/1698] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:44.464560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 857/1699] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:44 2022] PUT /identity/v3/projects/ea748f3b1bfe4433a4e4c1cde60ea741/users/1538406d73824e53ae6b8ec9a4a092af/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:44.468713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01c69e-89bb-4170-9c72-2471d1fa8eae None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:44.489619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b01c69e-89bb-4170-9c72-2471d1fa8eae None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:44.777170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28856194-661a-4335-a1a7-a66107fb7623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:44.778173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856194-661a-4335-a1a7-a66107fb7623 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:44.778471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856194-661a-4335-a1a7-a66107fb7623 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:44.779413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28856194-661a-4335-a1a7-a66107fb7623 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:44.780770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28856194-661a-4335-a1a7-a66107fb7623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:44.783178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28856194-661a-4335-a1a7-a66107fb7623 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:44.836161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 843/1700] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:44.840534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cf6de-c1f4-4838-a4d6-a18ef735ed9f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:44.857763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90cf6de-c1f4-4838-a4d6-a18ef735ed9f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:44.875353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:44.876467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:44.876862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:44.877210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] PATH_INFO: `/v3/projects/ea748f3b1bfe4433a4e4c1cde60ea741/users/1538406d73824e53ae6b8ec9a4a092af/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:44.882922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea748f3b1bfe4433a4e4c1cde60ea741, user_id=1538406d73824e53ae6b8ec9a4a092af, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:44.898564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:44.922669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 858/1701] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:44 2022] PUT /identity/v3/projects/ea748f3b1bfe4433a4e4c1cde60ea741/users/1538406d73824e53ae6b8ec9a4a092af/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:44.934348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50caa-d005-4e8c-ab12-7deff227ac87 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:44.934668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50caa-d005-4e8c-ab12-7deff227ac87 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:44.934997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50caa-d005-4e8c-ab12-7deff227ac87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:44.962625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:45.006084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d3b50caa-d005-4e8c-ab12-7deff227ac87 None None] MFA Rules not processed for user `1538406d73824e53ae6b8ec9a4a092af`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:45.007408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d3b50caa-d005-4e8c-ab12-7deff227ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:45.191760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 844/1702] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:45.202003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:45.204450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:45.270264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:45.345738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209e8d6-0d7b-4b20-9816-6a262aaa3f12 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:45.350697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08918108-0302-430c-aede-4fbabcffc92d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:45.360874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08918108-0302-430c-aede-4fbabcffc92d service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.361445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08918108-0302-430c-aede-4fbabcffc92d service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.361872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08918108-0302-430c-aede-4fbabcffc92d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.379062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08918108-0302-430c-aede-4fbabcffc92d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:45.381630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08918108-0302-430c-aede-4fbabcffc92d service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:45.429861 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 859/1703] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:45.728024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50caa-d005-4e8c-ab12-7deff227ac87 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:45.730332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d3b50caa-d005-4e8c-ab12-7deff227ac87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:45.754149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50caa-d005-4e8c-ab12-7deff227ac87 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:45.758983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf13e5-d46c-4406-ace3-04dfbe37eeb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:45.770883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb50b-bf9d-49ee-b17c-ad3b860641ac None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.771281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb50b-bf9d-49ee-b17c-ad3b860641ac None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.771664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb50b-bf9d-49ee-b17c-ad3b860641ac None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.773591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 860/1704] 10.222.0.40 () {54 vars in 912 bytes} [Mon Jul 25 05:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 05:59:45.782376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf13e5-d46c-4406-ace3-04dfbe37eeb9 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.782691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf13e5-d46c-4406-ace3-04dfbe37eeb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.784906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf13e5-d46c-4406-ace3-04dfbe37eeb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.795090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3dcf13e5-d46c-4406-ace3-04dfbe37eeb9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:45.799299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261c6d-d5b5-4701-85f1-abdebf74b857 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.799591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261c6d-d5b5-4701-85f1-abdebf74b857 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.806611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261c6d-d5b5-4701-85f1-abdebf74b857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.850749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-87261c6d-d5b5-4701-85f1-abdebf74b857 None None] MFA Rules not processed for user `b1638c28af8240439d061a3ae0ddc38d`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:45.850749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-87261c6d-d5b5-4701-85f1-abdebf74b857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:45.904034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf13e5-d46c-4406-ace3-04dfbe37eeb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:45.905812 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf13e5-d46c-4406-ace3-04dfbe37eeb9 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:45.919674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 845/1705] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:45.921669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91eca-a11b-4430-9289-8e356dcd5d51 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.923665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 861/1706] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 05:59:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:45.924243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91eca-a11b-4430-9289-8e356dcd5d51 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.924582 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a91eca-a11b-4430-9289-8e356dcd5d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.935679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261c6d-d5b5-4701-85f1-abdebf74b857 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:45.945959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261c6d-d5b5-4701-85f1-abdebf74b857 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:45.946928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-31a91eca-a11b-4430-9289-8e356dcd5d51 None None] MFA Rules not processed for user `8eedefd8d94047478663ceab8962d738`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:45.948610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-31a91eca-a11b-4430-9289-8e356dcd5d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:45.952321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:45.953234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:45.953684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:45.954150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:45.966801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:45.968655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:46.026785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 862/1707] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:46.076834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 846/1708] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:46.416685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:46.418037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:46.707719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac2e77-d48b-4f24-af01-f92f777a56f3 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:46.716119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:46.717049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:46.729422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:46.729763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] PATH_INFO: `/v3/projects/9abeac7667fc471e8dcd18f455bb328f` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:46.732882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=9abeac7667fc471e8dcd18f455bb328f)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:46.735869 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:46.925542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9abeac7667fc471e8dcd18f455bb328f', 'request_id': 'req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.009685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9abeac7667fc471e8dcd18f455bb328f', 'request_id': 'req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.012665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:47.022174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9abeac7667fc471e8dcd18f455bb328f', 'request_id': 'req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.044262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] The token cache is being invalidate because project 9abeac7667fc471e8dcd18f455bb328f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:47.044560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.045858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67c499fb-88e0-4d5c-9e4b-ae9a4bdec4ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.049607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 863/1709] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:46 2022] DELETE /identity/v3/projects/9abeac7667fc471e8dcd18f455bb328f => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:47.053444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a91eca-a11b-4430-9289-8e356dcd5d51 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:47.074629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a91eca-a11b-4430-9289-8e356dcd5d51 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:47.160721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:47.190295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:47.191294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:47.191579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:47.191862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] PATH_INFO: `/v3/users/e5e402a2ca3642c3b4c2812c87b5d36a` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:47.193178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e402a2ca3642c3b4c2812c87b5d36a)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:47.195237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:47.520640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c238-900a-4a7c-9066-1943d9658966 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:47.521130 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c238-900a-4a7c-9066-1943d9658966 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:47.521616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6c238-900a-4a7c-9066-1943d9658966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:47.583365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e402a2ca3642c3b4c2812c87b5d36a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.679711 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-8bf6c238-900a-4a7c-9066-1943d9658966 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:47.682028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8bf6c238-900a-4a7c-9066-1943d9658966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:47.746708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e402a2ca3642c3b4c2812c87b5d36a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.747339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e402a2ca3642c3b4c2812c87b5d36a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.781338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e402a2ca3642c3b4c2812c87b5d36a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.786049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e402a2ca3642c3b4c2812c87b5d36a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:47.838788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 864/1710] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 05:59:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:47.841854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 847/1711] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:47 2022] DELETE /identity/v3/users/e5e402a2ca3642c3b4c2812c87b5d36a => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:48.397070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6c238-900a-4a7c-9066-1943d9658966 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:48.397070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8bf6c238-900a-4a7c-9066-1943d9658966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:48.638509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6c238-900a-4a7c-9066-1943d9658966 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:48.736699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:48.801816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47709ab-7826-439f-be2a-c0adf1bcda4e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:48.801816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47709ab-7826-439f-be2a-c0adf1bcda4e service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:48.801816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47709ab-7826-439f-be2a-c0adf1bcda4e service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:48.801816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47709ab-7826-439f-be2a-c0adf1bcda4e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:48.803193 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:48.804856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:48.907298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47709ab-7826-439f-be2a-c0adf1bcda4e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:48.912145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47709ab-7826-439f-be2a-c0adf1bcda4e service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:48.943809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 865/1712] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 05:59:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 05:59:48.945942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35866ce-5213-4fa8-a6c1-5b37615775d0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:48.945942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35866ce-5213-4fa8-a6c1-5b37615775d0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:48.946293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35866ce-5213-4fa8-a6c1-5b37615775d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.013306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d35866ce-5213-4fa8-a6c1-5b37615775d0 None None] MFA Rules not processed for user `d485f9fc38a442dd9e1528024460fcca`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:49.018363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d35866ce-5213-4fa8-a6c1-5b37615775d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:49.085463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c610f77-f39b-4058-b9fc-0d6f923dcd42 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:49.105076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:49.111188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.111188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.111188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] PATH_INFO: `/v3/projects/8779958ba34f4c50a5c40935ac99ea6f` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.115772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8779958ba34f4c50a5c40935ac99ea6f)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:49.121311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:49.188331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 866/1713] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:49.416663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8779958ba34f4c50a5c40935ac99ea6f', 'request_id': 'req-dd12cc8f-156b-4f75-9f9a-8864e908b13d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:49.423636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:49.566709 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8779958ba34f4c50a5c40935ac99ea6f', 'request_id': 'req-dd12cc8f-156b-4f75-9f9a-8864e908b13d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:49.575887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8779958ba34f4c50a5c40935ac99ea6f', 'request_id': 'req-dd12cc8f-156b-4f75-9f9a-8864e908b13d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:49.628632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] The token cache is being invalidate because project 8779958ba34f4c50a5c40935ac99ea6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 05:59:49.639534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:49.648938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd12cc8f-156b-4f75-9f9a-8864e908b13d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:49.652189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 848/1714] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 05:59:48 2022] DELETE /identity/v3/projects/8779958ba34f4c50a5c40935ac99ea6f => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:49.664382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a9620-85f7-478c-a0be-d0485c249578 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.665041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a9620-85f7-478c-a0be-d0485c249578 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.665227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a9620-85f7-478c-a0be-d0485c249578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.708002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-605a9620-85f7-478c-a0be-d0485c249578 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:49.709828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-605a9620-85f7-478c-a0be-d0485c249578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:49.942417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 867/1715] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:49.952243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0bf11-0728-43d0-8fcd-8fd38be0881e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:49.952896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0bf11-0728-43d0-8fcd-8fd38be0881e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:49.953028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0bf11-0728-43d0-8fcd-8fd38be0881e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:49.990246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-74d0bf11-0728-43d0-8fcd-8fd38be0881e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:49.991478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-74d0bf11-0728-43d0-8fcd-8fd38be0881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:50.012995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 849/1716] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:50.017297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a9620-85f7-478c-a0be-d0485c249578 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.031507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a9620-85f7-478c-a0be-d0485c249578 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.053087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.057508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.062834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.064958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.068159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 05:59:50.068702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.076256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.115710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0bf11-0728-43d0-8fcd-8fd38be0881e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.126004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 868/1717] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 05:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 05:59:50.127239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0bf11-0728-43d0-8fcd-8fd38be0881e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.146801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64dc2e-85a3-4c02-8228-8a4fd5d7654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.149747 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64dc2e-85a3-4c02-8228-8a4fd5d7654e admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.153366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64dc2e-85a3-4c02-8228-8a4fd5d7654e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.153692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64dc2e-85a3-4c02-8228-8a4fd5d7654e admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.157222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64dc2e-85a3-4c02-8228-8a4fd5d7654e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.163287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64dc2e-85a3-4c02-8228-8a4fd5d7654e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:50.327196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 850/1718] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:50 2022] POST /identity/v3/projects => generated 367 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:50.332521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:50.359246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84d76-f6c0-4c95-8056-dd38afd8692f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:50.425101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:50.661242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e095bcd-a86a-4192-bf6e-3a1cc5d37ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:50.662141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e095bcd-a86a-4192-bf6e-3a1cc5d37ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:50.662499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e095bcd-a86a-4192-bf6e-3a1cc5d37ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:50.662959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e095bcd-a86a-4192-bf6e-3a1cc5d37ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:50.663827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e095bcd-a86a-4192-bf6e-3a1cc5d37ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:50.668211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e095bcd-a86a-4192-bf6e-3a1cc5d37ac4 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:51.081765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64dc2e-85a3-4c02-8228-8a4fd5d7654e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:51.098327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 869/1719] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:50 2022] POST /identity/v3/users => generated 284 bytes in 764 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:51.104996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64dc2e-85a3-4c02-8228-8a4fd5d7654e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:51.122730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689494d4-8286-478c-9947-ced94f7b6e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:51.123886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689494d4-8286-478c-9947-ced94f7b6e37 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:51.125934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689494d4-8286-478c-9947-ced94f7b6e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:51.129495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689494d4-8286-478c-9947-ced94f7b6e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:51.131601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689494d4-8286-478c-9947-ced94f7b6e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:51.140184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689494d4-8286-478c-9947-ced94f7b6e37 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:51.181076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 851/1720] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:51.185286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e095bcd-a86a-4192-bf6e-3a1cc5d37ac4 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:51.204180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e095bcd-a86a-4192-bf6e-3a1cc5d37ac4 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:51.220580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e67cf7-b993-47f1-b564-d65e53c85508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:51.223909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e67cf7-b993-47f1-b564-d65e53c85508 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:51.224234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e67cf7-b993-47f1-b564-d65e53c85508 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:51.226671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e67cf7-b993-47f1-b564-d65e53c85508 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:51.227722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e67cf7-b993-47f1-b564-d65e53c85508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:51.231994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e67cf7-b993-47f1-b564-d65e53c85508 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:51.252373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 870/1721] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:51.257299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689494d4-8286-478c-9947-ced94f7b6e37 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:51.291543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689494d4-8286-478c-9947-ced94f7b6e37 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:51.308161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d7a3-8079-4041-a1f1-64470f73545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:51.309495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d7a3-8079-4041-a1f1-64470f73545e admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:51.310135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d7a3-8079-4041-a1f1-64470f73545e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:51.311169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4923d7a3-8079-4041-a1f1-64470f73545e admin admin] PATH_INFO: `/v3/projects/470d5d01af964bfab3f11561aea31e88/users/2dce5459f28c4d04acea4acf1ec5857e/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:51.393895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923d7a3-8079-4041-a1f1-64470f73545e admin admin] RBAC: Authorizing `identity:create_grant(project_id=470d5d01af964bfab3f11561aea31e88, user_id=2dce5459f28c4d04acea4acf1ec5857e, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:51.402576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4923d7a3-8079-4041-a1f1-64470f73545e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:51.531856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 852/1722] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:51 2022] PUT /identity/v3/projects/470d5d01af964bfab3f11561aea31e88/users/2dce5459f28c4d04acea4acf1ec5857e/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:51.543992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f2523-1830-45d4-84ad-80291c8d9848 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:51.544394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f2523-1830-45d4-84ad-80291c8d9848 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:51.544932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f2523-1830-45d4-84ad-80291c8d9848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:51.616339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-035f2523-1830-45d4-84ad-80291c8d9848 None None] MFA Rules not processed for user `2dce5459f28c4d04acea4acf1ec5857e`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:51.617906 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-035f2523-1830-45d4-84ad-80291c8d9848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:51.794749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 871/1723] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:51.798549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d7a3-8079-4041-a1f1-64470f73545e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:52.023727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4923d7a3-8079-4041-a1f1-64470f73545e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:52.036731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30f1f0-84b7-47a7-8ae9-277c0392d279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:52.041987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30f1f0-84b7-47a7-8ae9-277c0392d279 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:52.041987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30f1f0-84b7-47a7-8ae9-277c0392d279 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:52.041987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30f1f0-84b7-47a7-8ae9-277c0392d279 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:52.042589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30f1f0-84b7-47a7-8ae9-277c0392d279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:52.047096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30f1f0-84b7-47a7-8ae9-277c0392d279 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:52.247776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 853/1724] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 05:59:51 2022] POST /identity/v3/projects => generated 367 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:52.249557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f2523-1830-45d4-84ad-80291c8d9848 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:52.268092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f2523-1830-45d4-84ad-80291c8d9848 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:52.492585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd79fce-243e-4930-a501-7fa2c75ed7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:52.494140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd79fce-243e-4930-a501-7fa2c75ed7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:52.494638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd79fce-243e-4930-a501-7fa2c75ed7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:52.495461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd79fce-243e-4930-a501-7fa2c75ed7a3 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:52.496388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd79fce-243e-4930-a501-7fa2c75ed7a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:52.499346 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd79fce-243e-4930-a501-7fa2c75ed7a3 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:52.686578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 872/1725] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 05:59:52 2022] POST /identity/v3/users => generated 290 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 05:59:52.695015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30f1f0-84b7-47a7-8ae9-277c0392d279 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:52.733867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30f1f0-84b7-47a7-8ae9-277c0392d279 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:52.757075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b46a1-10e9-407f-a126-51b6044ec1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:52.758972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b46a1-10e9-407f-a126-51b6044ec1af admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:52.760211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b46a1-10e9-407f-a126-51b6044ec1af admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:52.762449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b46a1-10e9-407f-a126-51b6044ec1af admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:52.763603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b46a1-10e9-407f-a126-51b6044ec1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:52.768302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b46a1-10e9-407f-a126-51b6044ec1af admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:52.826088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 854/1726] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 05:59:52.830380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd79fce-243e-4930-a501-7fa2c75ed7a3 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:52.854623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd79fce-243e-4930-a501-7fa2c75ed7a3 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:52.886752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312b301-85f8-47e2-b3aa-77831dae9296 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:52.886752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312b301-85f8-47e2-b3aa-77831dae9296 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:52.886752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312b301-85f8-47e2-b3aa-77831dae9296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:52.896337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf74dcf-b0d3-46b8-b9f2-ab2926c2b859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:52.900274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74dcf-b0d3-46b8-b9f2-ab2926c2b859 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:52.901296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74dcf-b0d3-46b8-b9f2-ab2926c2b859 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:52.901849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf74dcf-b0d3-46b8-b9f2-ab2926c2b859 admin admin] PATH_INFO: `/v3/projects/00980b654c7d439fb1e43e746df54f38/users/eca17e44a44a4d109b593f80272c232a/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:52.946171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:52.953967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3312b301-85f8-47e2-b3aa-77831dae9296 None None] MFA Rules not processed for user `b1638c28af8240439d061a3ae0ddc38d`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:52.957677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3312b301-85f8-47e2-b3aa-77831dae9296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:52.982759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74dcf-b0d3-46b8-b9f2-ab2926c2b859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00980b654c7d439fb1e43e746df54f38, user_id=eca17e44a44a4d109b593f80272c232a, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:52.992031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf74dcf-b0d3-46b8-b9f2-ab2926c2b859 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:53.110142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 873/1727] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:52 2022] PUT /identity/v3/projects/00980b654c7d439fb1e43e746df54f38/users/eca17e44a44a4d109b593f80272c232a/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:53.114030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf74dcf-b0d3-46b8-b9f2-ab2926c2b859 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:53.137409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:53.154126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 855/1728] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Jul 25 05:59:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 05:59:53.384329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf74dcf-b0d3-46b8-b9f2-ab2926c2b859 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:53.428407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beb187-1a70-4676-80d2-398bb6f1f2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:53.429122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beb187-1a70-4676-80d2-398bb6f1f2fb admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:53.429415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beb187-1a70-4676-80d2-398bb6f1f2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:53.429693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beb187-1a70-4676-80d2-398bb6f1f2fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:53.430721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beb187-1a70-4676-80d2-398bb6f1f2fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:53.444231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beb187-1a70-4676-80d2-398bb6f1f2fb admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:53.465077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 874/1729] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 05:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 05:59:53.465229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312b301-85f8-47e2-b3aa-77831dae9296 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:53.489476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312b301-85f8-47e2-b3aa-77831dae9296 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:53.509935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:53.511965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:53.515621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:53.515621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] PATH_INFO: `/v3/projects/00980b654c7d439fb1e43e746df54f38/users/eca17e44a44a4d109b593f80272c232a/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:53.517585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=00980b654c7d439fb1e43e746df54f38, user_id=eca17e44a44a4d109b593f80272c232a, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:53.524233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:53.569100 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 856/1730] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 05:59:53 2022] PUT /identity/v3/projects/00980b654c7d439fb1e43e746df54f38/users/eca17e44a44a4d109b593f80272c232a/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 05:59:53.574991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad0ffc-0483-4d2c-b07d-4650b3c2de9e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:53.575164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad0ffc-0483-4d2c-b07d-4650b3c2de9e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:53.582244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad0ffc-0483-4d2c-b07d-4650b3c2de9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:53.621465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9dad0ffc-0483-4d2c-b07d-4650b3c2de9e None None] MFA Rules not processed for user `eca17e44a44a4d109b593f80272c232a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:53.625515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9dad0ffc-0483-4d2c-b07d-4650b3c2de9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:53.810549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 875/1731] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 05:59:53.824987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:53.826873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:53.930535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d1462-3fbf-4075-ba27-2211f937d2af admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:53.935302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:53.935981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:53.936333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:53.937260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:53.970355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:53.972206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:53.996201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 857/1732] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:55.923731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad0ffc-0483-4d2c-b07d-4650b3c2de9e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:55.926717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9dad0ffc-0483-4d2c-b07d-4650b3c2de9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:55.970307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:55.970307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:56.185762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad0ffc-0483-4d2c-b07d-4650b3c2de9e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.192013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.196081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.197104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.198871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.204703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf56aa2-a25a-4247-8bc5-294bcc80990b service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.224234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f92a4-358d-4ce2-8425-f9fcb9779145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.225048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f92a4-358d-4ce2-8425-f9fcb9779145 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.225373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f92a4-358d-4ce2-8425-f9fcb9779145 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.230319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f92a4-358d-4ce2-8425-f9fcb9779145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.254988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f92a4-358d-4ce2-8425-f9fcb9779145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.257003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f92a4-358d-4ce2-8425-f9fcb9779145 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.262248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.267637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.280585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 858/1733] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 05:59:56.281585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f92a4-358d-4ce2-8425-f9fcb9779145 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.314528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f92a4-358d-4ce2-8425-f9fcb9779145 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.325293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 876/1734] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:56.326757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:56.327447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a83cf-6f95-4ade-9b87-2a2c3ee3bc6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.328138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a83cf-6f95-4ade-9b87-2a2c3ee3bc6b service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.328438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a83cf-6f95-4ade-9b87-2a2c3ee3bc6b service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.328776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a83cf-6f95-4ade-9b87-2a2c3ee3bc6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.329485 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 05:59:56.361525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a83cf-6f95-4ade-9b87-2a2c3ee3bc6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.366770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a83cf-6f95-4ade-9b87-2a2c3ee3bc6b service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.388306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a14715-dad3-4c09-bc99-3c8bbd8fa47e service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:56.419773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 859/1735] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:56.425398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee071f8-01b5-49b7-a066-2179f36aa28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:56.427698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee071f8-01b5-49b7-a066-2179f36aa28d service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:56.428500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee071f8-01b5-49b7-a066-2179f36aa28d service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:56.428712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee071f8-01b5-49b7-a066-2179f36aa28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:56.480848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee071f8-01b5-49b7-a066-2179f36aa28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:56.483982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee071f8-01b5-49b7-a066-2179f36aa28d service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:56.497880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:56.502240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 877/1736] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 05:59:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 05:59:59.237159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a83cf-6f95-4ade-9b87-2a2c3ee3bc6b service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 05:59:59.494004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a83cf-6f95-4ade-9b87-2a2c3ee3bc6b service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 05:59:59.516472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 05:59:59.517877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.518374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.518826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] PATH_INFO: `/v3/users/2dce5459f28c4d04acea4acf1ec5857e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.526555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dce5459f28c4d04acea4acf1ec5857e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 05:59:59.537313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 05:59:59.601042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 05:59:59.802117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dce5459f28c4d04acea4acf1ec5857e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:59.866424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dce5459f28c4d04acea4acf1ec5857e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:59.868182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dce5459f28c4d04acea4acf1ec5857e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:59.881266 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dce5459f28c4d04acea4acf1ec5857e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:59.883573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dce5459f28c4d04acea4acf1ec5857e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 05:59:59.914409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 860/1737] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 05:59:59 2022] DELETE /identity/v3/users/2dce5459f28c4d04acea4acf1ec5857e => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 05:59:59.919329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc6533-2060-4ea8-b549-1cc59c542232 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 05:59:59.920381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc6533-2060-4ea8-b549-1cc59c542232 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 05:59:59.920807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc6533-2060-4ea8-b549-1cc59c542232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 05:59:59.973444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-74fc6533-2060-4ea8-b549-1cc59c542232 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 05:59:59.974696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-74fc6533-2060-4ea8-b549-1cc59c542232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:00.074529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 878/1738] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 05:59:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:00.124371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:00.126185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:00.330174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9d414-75dc-4241-ba10-be44a934791a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:00.369099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f19cec-0b2c-4a51-a6d8-a038cc06197c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:00.373433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19cec-0b2c-4a51-a6d8-a038cc06197c service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:00.373808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19cec-0b2c-4a51-a6d8-a038cc06197c service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:00.374092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f19cec-0b2c-4a51-a6d8-a038cc06197c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:00.418840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f19cec-0b2c-4a51-a6d8-a038cc06197c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:00.423556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f19cec-0b2c-4a51-a6d8-a038cc06197c service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:00.486190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 861/1739] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:00.819395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d77b73-e534-4fc2-bb1b-374d22c0f3b6 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:00.820007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d77b73-e534-4fc2-bb1b-374d22c0f3b6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:00.820508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d77b73-e534-4fc2-bb1b-374d22c0f3b6 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:00.822854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 879/1740] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 06:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:00:00.879723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7b593-3a2d-4e17-911f-7312c3fac8bb None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:00.880223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7b593-3a2d-4e17-911f-7312c3fac8bb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:00.880603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7b593-3a2d-4e17-911f-7312c3fac8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:00.933598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ace7b593-3a2d-4e17-911f-7312c3fac8bb None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:00.935169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ace7b593-3a2d-4e17-911f-7312c3fac8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:01.171613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 862/1741] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 06:00:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:01.192193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d77b73-e534-4fc2-bb1b-374d22c0f3b6 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:01.195143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-98d77b73-e534-4fc2-bb1b-374d22c0f3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:01.229043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d77b73-e534-4fc2-bb1b-374d22c0f3b6 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:01.237703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:01.242055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:01.242055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:01.242055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] PATH_INFO: `/v3/users/6c6486c089e14df29cf450f033b9e317` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:01.242055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c6486c089e14df29cf450f033b9e317)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:01.245340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:01.470821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c6486c089e14df29cf450f033b9e317'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.534826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c6486c089e14df29cf450f033b9e317'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.535797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c6486c089e14df29cf450f033b9e317'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.557330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c6486c089e14df29cf450f033b9e317'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.560719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c6486c089e14df29cf450f033b9e317'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:01.622733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 880/1742] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:01 2022] DELETE /identity/v3/users/6c6486c089e14df29cf450f033b9e317 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:01.628389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a88b47-0221-423d-aebc-ffcdc75d641d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:01.629256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a88b47-0221-423d-aebc-ffcdc75d641d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:01.629686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a88b47-0221-423d-aebc-ffcdc75d641d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:01.663752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-04a88b47-0221-423d-aebc-ffcdc75d641d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:01.665462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-04a88b47-0221-423d-aebc-ffcdc75d641d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:01.777276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:01.838771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 863/1743] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:01.867831 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:01.870141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:02.043517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac13f2-e1e3-4de4-b072-f29b8373e63b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:02.058576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:02.062228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.062228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.062228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.077376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:02.078844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:02.095676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 881/1744] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:02.113025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69414389-5add-4fdb-811f-7a3fca2c46a6 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.118763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69414389-5add-4fdb-811f-7a3fca2c46a6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.119696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69414389-5add-4fdb-811f-7a3fca2c46a6 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.130416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 864/1745] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 06:00:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:00:02.249024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:02.250509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:02.289607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:02.356650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50231-06f1-49f9-a4f8-374b13e9b98c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.356910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50231-06f1-49f9-a4f8-374b13e9b98c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.357425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe50231-06f1-49f9-a4f8-374b13e9b98c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.394953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8fe50231-06f1-49f9-a4f8-374b13e9b98c None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:02.420407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8fe50231-06f1-49f9-a4f8-374b13e9b98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:02.430714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fcea9-2eff-447b-b02c-68477a09e7c5 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:02.450260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:02.450487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 865/1746] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 06:00:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:02.450941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:02.451231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:02.451516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:02.458993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:02.490516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:02.492367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:02.518148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 882/1747] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:02.958177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe50231-06f1-49f9-a4f8-374b13e9b98c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:02.995063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe50231-06f1-49f9-a4f8-374b13e9b98c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:03.007828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:03.009527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:03.010016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:03.013668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:03.057575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:03.059651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:03.072719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 866/1748] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:04.564781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:04.566249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:04.580231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ebc4a-0e2c-4494-85ac-5cc98d148906 service nova] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:04.586941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:04.587969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:04.588411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:04.589045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] PATH_INFO: `/v3/projects/470d5d01af964bfab3f11561aea31e88` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:04.590473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] RBAC: Authorizing `identity:delete_project(project_id=470d5d01af964bfab3f11561aea31e88)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:04.592959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:04.697209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:04.698906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:04.726394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2ec5d-c8a2-4991-a26b-ab08032bf381 service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:04.741202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470d5d01af964bfab3f11561aea31e88', 'request_id': 'req-20d26b2d-a464-40ce-8b85-b74c1559c30d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:04.830502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:04.831376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:04.831658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:04.831929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] PATH_INFO: `/v3/projects/78324bbf84ef41a0867446a5f8e6925a` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:04.833160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78324bbf84ef41a0867446a5f8e6925a)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:04.835072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:04.841920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470d5d01af964bfab3f11561aea31e88', 'request_id': 'req-20d26b2d-a464-40ce-8b85-b74c1559c30d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:04.850160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470d5d01af964bfab3f11561aea31e88', 'request_id': 'req-20d26b2d-a464-40ce-8b85-b74c1559c30d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:04.878716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] The token cache is being invalidate because project 470d5d01af964bfab3f11561aea31e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:04.880294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:04.880294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:04.881912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 883/1749] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:04 2022] DELETE /identity/v3/projects/470d5d01af964bfab3f11561aea31e88 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:04.886918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:04.888589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:04.919192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d26b2d-a464-40ce-8b85-b74c1559c30d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:04.929566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:04.932035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:04.932811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:04.933851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] PATH_INFO: `/v3/users/eca17e44a44a4d109b593f80272c232a` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:04.935894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca17e44a44a4d109b593f80272c232a)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:04.941272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:04.968381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78324bbf84ef41a0867446a5f8e6925a', 'request_id': 'req-411a0b1a-391d-41ee-b789-12d4294223e1'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.060385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78324bbf84ef41a0867446a5f8e6925a', 'request_id': 'req-411a0b1a-391d-41ee-b789-12d4294223e1'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.071605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78324bbf84ef41a0867446a5f8e6925a', 'request_id': 'req-411a0b1a-391d-41ee-b789-12d4294223e1'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.086547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] The token cache is being invalidate because project 78324bbf84ef41a0867446a5f8e6925a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:05.088004 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.088813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.090786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 867/1750] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:04 2022] DELETE /identity/v3/projects/78324bbf84ef41a0867446a5f8e6925a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:05.100441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:05.101973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:05.148861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:05.167814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a0b1a-391d-41ee-b789-12d4294223e1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:05.172626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:05.173503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:05.173785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:05.174066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] PATH_INFO: `/v3/users/1538406d73824e53ae6b8ec9a4a092af` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:05.175257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1538406d73824e53ae6b8ec9a4a092af)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:05.177226 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:05.209662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca17e44a44a4d109b593f80272c232a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.340380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca17e44a44a4d109b593f80272c232a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.341059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca17e44a44a4d109b593f80272c232a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.353387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca17e44a44a4d109b593f80272c232a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.354248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca17e44a44a4d109b593f80272c232a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.368389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:05.376572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 884/1751] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:04 2022] DELETE /identity/v3/users/eca17e44a44a4d109b593f80272c232a => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:05.420640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1538406d73824e53ae6b8ec9a4a092af'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.541274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1538406d73824e53ae6b8ec9a4a092af'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.541988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1538406d73824e53ae6b8ec9a4a092af'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.549692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1538406d73824e53ae6b8ec9a4a092af'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.550477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1538406d73824e53ae6b8ec9a4a092af'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:05.569809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 868/1752] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:05 2022] DELETE /identity/v3/users/1538406d73824e53ae6b8ec9a4a092af => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:05.891526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:05.893216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:06.009116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9cc9d-08cf-4fa2-b272-43ab610def11 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.075541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.076580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.076867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.077149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] PATH_INFO: `/v3/projects/00980b654c7d439fb1e43e746df54f38` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.078459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00980b654c7d439fb1e43e746df54f38)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.080426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.103613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.107144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:06.125729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138c8d-b5d5-4cc2-aa56-b37692f7f5a7 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.131628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.135299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.135299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.135299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] PATH_INFO: `/v3/projects/ea748f3b1bfe4433a4e4c1cde60ea741` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.135299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea748f3b1bfe4433a4e4c1cde60ea741)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.136544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.186527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00980b654c7d439fb1e43e746df54f38', 'request_id': 'req-b03fb2b4-e090-420d-982d-8fe84a6171d5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.228212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea748f3b1bfe4433a4e4c1cde60ea741', 'request_id': 'req-dce77298-c263-4e76-b024-8ab1cfdfe4bd'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.260097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00980b654c7d439fb1e43e746df54f38', 'request_id': 'req-b03fb2b4-e090-420d-982d-8fe84a6171d5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.266471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00980b654c7d439fb1e43e746df54f38', 'request_id': 'req-b03fb2b4-e090-420d-982d-8fe84a6171d5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.279718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] The token cache is being invalidate because project 00980b654c7d439fb1e43e746df54f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:06.280014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.280706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b03fb2b4-e090-420d-982d-8fe84a6171d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.282281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 885/1753] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:05 2022] DELETE /identity/v3/projects/00980b654c7d439fb1e43e746df54f38 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:06.287189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d7672-e1a0-4ea5-9923-70dc65749a46 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.287392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d7672-e1a0-4ea5-9923-70dc65749a46 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.287679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283d7672-e1a0-4ea5-9923-70dc65749a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.307696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-283d7672-e1a0-4ea5-9923-70dc65749a46 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:06.308959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-283d7672-e1a0-4ea5-9923-70dc65749a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:06.339219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:06.345629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea748f3b1bfe4433a4e4c1cde60ea741', 'request_id': 'req-dce77298-c263-4e76-b024-8ab1cfdfe4bd'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.352686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea748f3b1bfe4433a4e4c1cde60ea741', 'request_id': 'req-dce77298-c263-4e76-b024-8ab1cfdfe4bd'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.364543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] The token cache is being invalidate because project ea748f3b1bfe4433a4e4c1cde60ea741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:06.364971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.365866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dce77298-c263-4e76-b024-8ab1cfdfe4bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:06.367495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 869/1754] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:06 2022] DELETE /identity/v3/projects/ea748f3b1bfe4433a4e4c1cde60ea741 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:06.370900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 886/1755] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:06.372527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd184a2-18e7-4867-ab43-6edf1f2ee95b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.372884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd184a2-18e7-4867-ab43-6edf1f2ee95b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.373232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd184a2-18e7-4867-ab43-6edf1f2ee95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.379350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683204e-8a1f-420a-a37d-47375a1dd3fa None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.379748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683204e-8a1f-420a-a37d-47375a1dd3fa None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.380036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683204e-8a1f-420a-a37d-47375a1dd3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.393271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1fd184a2-18e7-4867-ab43-6edf1f2ee95b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:06.394491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1fd184a2-18e7-4867-ab43-6edf1f2ee95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:06.399759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1683204e-8a1f-420a-a37d-47375a1dd3fa None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:06.401254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1683204e-8a1f-420a-a37d-47375a1dd3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:06.403716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 870/1756] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:06.411317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a834b4-3b9c-412c-87a8-be9834343499 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.411638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a834b4-3b9c-412c-87a8-be9834343499 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.411919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a834b4-3b9c-412c-87a8-be9834343499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.413824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 887/1757] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:06.417851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683204e-8a1f-420a-a37d-47375a1dd3fa None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.426905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683204e-8a1f-420a-a37d-47375a1dd3fa None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.429783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-85a834b4-3b9c-412c-87a8-be9834343499 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:06.431416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-85a834b4-3b9c-412c-87a8-be9834343499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:06.431894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.432628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.432920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.433204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.433940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:06.434252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.435848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.440105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 871/1758] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:06.444512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a834b4-3b9c-412c-87a8-be9834343499 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.451945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 888/1759] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:06.453029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.454273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a834b4-3b9c-412c-87a8-be9834343499 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.460443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.461104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.461926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.462215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.462950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:06.463259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.463927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c890c-7c7d-4263-8b63-684309b69ded admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.464889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.469759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2f6c6-8341-40d6-9cca-85b2de97fb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.470486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2f6c6-8341-40d6-9cca-85b2de97fb94 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.470888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2f6c6-8341-40d6-9cca-85b2de97fb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.471269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2f6c6-8341-40d6-9cca-85b2de97fb94 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.472778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2f6c6-8341-40d6-9cca-85b2de97fb94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.475416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2f6c6-8341-40d6-9cca-85b2de97fb94 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.476720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 872/1760] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:06.481392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.491803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b6e24-b304-4430-b193-198d011fcb3b admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.496755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01df16-89a5-4af7-bfb5-ba64f7db200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.497405 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01df16-89a5-4af7-bfb5-ba64f7db200f admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.497693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01df16-89a5-4af7-bfb5-ba64f7db200f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.497970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01df16-89a5-4af7-bfb5-ba64f7db200f admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.498772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01df16-89a5-4af7-bfb5-ba64f7db200f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.500729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01df16-89a5-4af7-bfb5-ba64f7db200f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.570626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 889/1761] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:06.575565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2f6c6-8341-40d6-9cca-85b2de97fb94 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.631084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:06.635691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2f6c6-8341-40d6-9cca-85b2de97fb94 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.642882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 873/1762] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:06.643018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbb63a-f098-44ea-b975-1a0f544e6852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.643018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbb63a-f098-44ea-b975-1a0f544e6852 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.643018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbb63a-f098-44ea-b975-1a0f544e6852 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.643018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbb63a-f098-44ea-b975-1a0f544e6852 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.643304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbb63a-f098-44ea-b975-1a0f544e6852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.643777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01df16-89a5-4af7-bfb5-ba64f7db200f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.644596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbb63a-f098-44ea-b975-1a0f544e6852 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.704733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01df16-89a5-4af7-bfb5-ba64f7db200f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.709351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3f43b-f09b-4b50-bc6e-3dcf78b2cb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.710035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f43b-f09b-4b50-bc6e-3dcf78b2cb4f admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.710319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f43b-f09b-4b50-bc6e-3dcf78b2cb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.710597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3f43b-f09b-4b50-bc6e-3dcf78b2cb4f admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.712144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3f43b-f09b-4b50-bc6e-3dcf78b2cb4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.714190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3f43b-f09b-4b50-bc6e-3dcf78b2cb4f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.728265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 890/1763] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:06.729205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbb63a-f098-44ea-b975-1a0f544e6852 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.741471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbb63a-f098-44ea-b975-1a0f544e6852 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.748482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4241613-a78a-40e6-a901-1c855f8bdf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.749448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4241613-a78a-40e6-a901-1c855f8bdf82 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.750415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4241613-a78a-40e6-a901-1c855f8bdf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.750863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4241613-a78a-40e6-a901-1c855f8bdf82 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.751649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4241613-a78a-40e6-a901-1c855f8bdf82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.756685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4241613-a78a-40e6-a901-1c855f8bdf82 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.768168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 891/1764] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:06.772767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4241613-a78a-40e6-a901-1c855f8bdf82 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.787020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4241613-a78a-40e6-a901-1c855f8bdf82 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.791624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c24a-f20c-4cb1-829e-800aedc2b13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.792342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c24a-f20c-4cb1-829e-800aedc2b13e admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.792650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c24a-f20c-4cb1-829e-800aedc2b13e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.792911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0c24a-f20c-4cb1-829e-800aedc2b13e admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.793685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0c24a-f20c-4cb1-829e-800aedc2b13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.796523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0c24a-f20c-4cb1-829e-800aedc2b13e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.809944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 874/1765] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:06.814424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3f43b-f09b-4b50-bc6e-3dcf78b2cb4f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.827997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3f43b-f09b-4b50-bc6e-3dcf78b2cb4f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.831951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 892/1766] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:06.833235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c24a-f20c-4cb1-829e-800aedc2b13e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.847295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1d6cb-be66-4fa3-bfe9-64f30199523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.850991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d6cb-be66-4fa3-bfe9-64f30199523b admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.850991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d6cb-be66-4fa3-bfe9-64f30199523b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.850991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d6cb-be66-4fa3-bfe9-64f30199523b admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.850991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1d6cb-be66-4fa3-bfe9-64f30199523b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.850991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a1d6cb-be66-4fa3-bfe9-64f30199523b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.852973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0c24a-f20c-4cb1-829e-800aedc2b13e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.859631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d93803-8e91-40ed-8f24-52dca374ea77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.860585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93803-8e91-40ed-8f24-52dca374ea77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.860864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93803-8e91-40ed-8f24-52dca374ea77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.861428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d93803-8e91-40ed-8f24-52dca374ea77 admin admin] PATH_INFO: `/v3/projects/f153234f1bd14f3b8f102795135d17ac/users/508e32cc95934fdf90c4ff1ae0fe393d/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.879994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 875/1767] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:06.881641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d93803-8e91-40ed-8f24-52dca374ea77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f153234f1bd14f3b8f102795135d17ac, user_id=508e32cc95934fdf90c4ff1ae0fe393d, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.888470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1d6cb-be66-4fa3-bfe9-64f30199523b admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.888630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d93803-8e91-40ed-8f24-52dca374ea77 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.897920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1d6cb-be66-4fa3-bfe9-64f30199523b admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.905447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c8321-d27d-437e-9611-a7bdcd167e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.907220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8321-d27d-437e-9611-a7bdcd167e62 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.911121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8321-d27d-437e-9611-a7bdcd167e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.911121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c8321-d27d-437e-9611-a7bdcd167e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.911121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c8321-d27d-437e-9611-a7bdcd167e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.911893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c8321-d27d-437e-9611-a7bdcd167e62 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:06.926239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 876/1768] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:06.930983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c8321-d27d-437e-9611-a7bdcd167e62 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:06.942409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c8321-d27d-437e-9611-a7bdcd167e62 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:06.953789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92847fc-4d56-47f2-8502-c585d007eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:06.954422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 893/1769] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:06 2022] PUT /identity/v3/projects/f153234f1bd14f3b8f102795135d17ac/users/508e32cc95934fdf90c4ff1ae0fe393d/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:06.964565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92847fc-4d56-47f2-8502-c585d007eff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.965021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92847fc-4d56-47f2-8502-c585d007eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.965324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92847fc-4d56-47f2-8502-c585d007eff6 admin admin] PATH_INFO: `/v3/projects/f672d88e89f1451d98118b5d696893bb/users/b1e0a0d928c14ae295c296f5aa7d6d59/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.966145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27971b-196f-41f6-8cea-211919d50cab None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:06.966516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27971b-196f-41f6-8cea-211919d50cab None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:06.966896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27971b-196f-41f6-8cea-211919d50cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:06.989420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92847fc-4d56-47f2-8502-c585d007eff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f672d88e89f1451d98118b5d696893bb, user_id=b1e0a0d928c14ae295c296f5aa7d6d59, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:06.994015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92847fc-4d56-47f2-8502-c585d007eff6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.043641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ea27971b-196f-41f6-8cea-211919d50cab None None] MFA Rules not processed for user `508e32cc95934fdf90c4ff1ae0fe393d`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:07.045555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ea27971b-196f-41f6-8cea-211919d50cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:07.054158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 877/1770] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:06 2022] PUT /identity/v3/projects/f672d88e89f1451d98118b5d696893bb/users/b1e0a0d928c14ae295c296f5aa7d6d59/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:07.068141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4e5be-bf35-4002-937d-4be4f0c16da3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.068483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4e5be-bf35-4002-937d-4be4f0c16da3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.068850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4e5be-bf35-4002-937d-4be4f0c16da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.082084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:07.113929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-add4e5be-bf35-4002-937d-4be4f0c16da3 None None] MFA Rules not processed for user `b1e0a0d928c14ae295c296f5aa7d6d59`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:07.115558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-add4e5be-bf35-4002-937d-4be4f0c16da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:07.159829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 894/1771] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:07.164250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27971b-196f-41f6-8cea-211919d50cab None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.258065 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 878/1772] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:07.263658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4e5be-bf35-4002-937d-4be4f0c16da3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.318537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:07.325326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27971b-196f-41f6-8cea-211919d50cab None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.332220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09d3c4-71b1-41a9-a013-a66706633000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:07.333478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d3c4-71b1-41a9-a013-a66706633000 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.334001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d3c4-71b1-41a9-a013-a66706633000 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.340043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09d3c4-71b1-41a9-a013-a66706633000 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.340043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09d3c4-71b1-41a9-a013-a66706633000 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:07.340043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09d3c4-71b1-41a9-a013-a66706633000 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.355789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4e5be-bf35-4002-937d-4be4f0c16da3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.363053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d3ec8-f35a-4550-8ea6-2f3234f26c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:07.363752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d3ec8-f35a-4550-8ea6-2f3234f26c83 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.364039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d3ec8-f35a-4550-8ea6-2f3234f26c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.364345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d3ec8-f35a-4550-8ea6-2f3234f26c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.365052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d3ec8-f35a-4550-8ea6-2f3234f26c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:07.369197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d3ec8-f35a-4550-8ea6-2f3234f26c83 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.458749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 895/1773] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:07 2022] POST /identity/v3/projects => generated 389 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:07.463061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09d3c4-71b1-41a9-a013-a66706633000 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.476659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 879/1774] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:07 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:07.479991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d3ec8-f35a-4550-8ea6-2f3234f26c83 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.597854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09d3c4-71b1-41a9-a013-a66706633000 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.607954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d6414-1bd5-4d64-bc20-4c49281cd4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:07.608775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d6414-1bd5-4d64-bc20-4c49281cd4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.609133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d6414-1bd5-4d64-bc20-4c49281cd4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.609601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d6414-1bd5-4d64-bc20-4c49281cd4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.611949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d3ec8-f35a-4550-8ea6-2f3234f26c83 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.614645 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d6414-1bd5-4d64-bc20-4c49281cd4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:07.619333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d6414-1bd5-4d64-bc20-4c49281cd4e5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.627174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed275-0b4c-4df8-8fc1-5b88cdc2cdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:07.627989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed275-0b4c-4df8-8fc1-5b88cdc2cdf6 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.628530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed275-0b4c-4df8-8fc1-5b88cdc2cdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.628815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ed275-0b4c-4df8-8fc1-5b88cdc2cdf6 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.629708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed275-0b4c-4df8-8fc1-5b88cdc2cdf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:07.631748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ed275-0b4c-4df8-8fc1-5b88cdc2cdf6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.685510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 896/1775] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:07 2022] POST /identity/v3/users => generated 301 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:07.689515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d6414-1bd5-4d64-bc20-4c49281cd4e5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.707773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 880/1776] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:07 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:07.711971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d6414-1bd5-4d64-bc20-4c49281cd4e5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.712262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed275-0b4c-4df8-8fc1-5b88cdc2cdf6 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.719913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49968893-2bf7-4eca-a445-af07fc19d637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:07.720902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49968893-2bf7-4eca-a445-af07fc19d637 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.721470 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49968893-2bf7-4eca-a445-af07fc19d637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.721852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49968893-2bf7-4eca-a445-af07fc19d637 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.722782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49968893-2bf7-4eca-a445-af07fc19d637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:07.724799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49968893-2bf7-4eca-a445-af07fc19d637 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.725207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ed275-0b4c-4df8-8fc1-5b88cdc2cdf6 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.735360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12ca1a-7eb2-4238-8b1b-fddb0c9b3612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:07.736155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ca1a-7eb2-4238-8b1b-fddb0c9b3612 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.736420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ca1a-7eb2-4238-8b1b-fddb0c9b3612 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.736753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ca1a-7eb2-4238-8b1b-fddb0c9b3612 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.737457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12ca1a-7eb2-4238-8b1b-fddb0c9b3612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:07.739099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12ca1a-7eb2-4238-8b1b-fddb0c9b3612 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.744242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 897/1777] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:07.745368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49968893-2bf7-4eca-a445-af07fc19d637 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.756300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 881/1778] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:07.760583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12ca1a-7eb2-4238-8b1b-fddb0c9b3612 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.761317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49968893-2bf7-4eca-a445-af07fc19d637 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.767705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd4add-ec40-4a32-8663-1238abc6dd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:07.767705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd4add-ec40-4a32-8663-1238abc6dd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.768164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd4add-ec40-4a32-8663-1238abc6dd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.768625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd4add-ec40-4a32-8663-1238abc6dd3a admin admin] PATH_INFO: `/v3/projects/09b04936e63a4f3fa923ef563857e4e0/users/48f60114d1124fa0a9b2d57c7f3e2923/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.777305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12ca1a-7eb2-4238-8b1b-fddb0c9b3612 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.782524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18adde-bb59-4e22-9821-b2088541a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:07.783423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18adde-bb59-4e22-9821-b2088541a06c admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:07.783706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18adde-bb59-4e22-9821-b2088541a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:07.783983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18adde-bb59-4e22-9821-b2088541a06c admin admin] PATH_INFO: `/v3/projects/dc5e4f36d6a843b0804bfb72ffdee8b6/users/f6edb6c54e7f48dca958e09f3b6bf6f5/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:07.789132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd4add-ec40-4a32-8663-1238abc6dd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b04936e63a4f3fa923ef563857e4e0, user_id=48f60114d1124fa0a9b2d57c7f3e2923, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:07.791912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd4add-ec40-4a32-8663-1238abc6dd3a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.799964 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18adde-bb59-4e22-9821-b2088541a06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5e4f36d6a843b0804bfb72ffdee8b6, user_id=f6edb6c54e7f48dca958e09f3b6bf6f5, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:07.802660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18adde-bb59-4e22-9821-b2088541a06c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:07.875688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 898/1779] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:07 2022] PUT /identity/v3/projects/09b04936e63a4f3fa923ef563857e4e0/users/48f60114d1124fa0a9b2d57c7f3e2923/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:07.880326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd4add-ec40-4a32-8663-1238abc6dd3a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:07.889354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:07.952795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 882/1780] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:07 2022] PUT /identity/v3/projects/dc5e4f36d6a843b0804bfb72ffdee8b6/users/f6edb6c54e7f48dca958e09f3b6bf6f5/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:07.952993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd4add-ec40-4a32-8663-1238abc6dd3a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:07.953085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18adde-bb59-4e22-9821-b2088541a06c admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:08.012014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d577e-afb5-4d05-9a64-65e35d245033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:08.012795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d577e-afb5-4d05-9a64-65e35d245033 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.013043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d577e-afb5-4d05-9a64-65e35d245033 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.013264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5d577e-afb5-4d05-9a64-65e35d245033 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.014000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d577e-afb5-4d05-9a64-65e35d245033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:08.015666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5d577e-afb5-4d05-9a64-65e35d245033 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:08.026833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 899/1781] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:08.198738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18adde-bb59-4e22-9821-b2088541a06c admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:08.198738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af958e7b-e317-479d-9238-241b22c10da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:08.198738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af958e7b-e317-479d-9238-241b22c10da3 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.198738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af958e7b-e317-479d-9238-241b22c10da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.199514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af958e7b-e317-479d-9238-241b22c10da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.199514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af958e7b-e317-479d-9238-241b22c10da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:08.199514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af958e7b-e317-479d-9238-241b22c10da3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:08.199514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 883/1782] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:08.200501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d577e-afb5-4d05-9a64-65e35d245033 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:08.200774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af958e7b-e317-479d-9238-241b22c10da3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:08.200774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af958e7b-e317-479d-9238-241b22c10da3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:08.201187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5d577e-afb5-4d05-9a64-65e35d245033 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:08.201970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca713fc-3462-4d26-8714-35dc8cdb4688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:08.203045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca713fc-3462-4d26-8714-35dc8cdb4688 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.203502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca713fc-3462-4d26-8714-35dc8cdb4688 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.203856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca713fc-3462-4d26-8714-35dc8cdb4688 admin admin] PATH_INFO: `/v3/projects/dc5e4f36d6a843b0804bfb72ffdee8b6/users/f6edb6c54e7f48dca958e09f3b6bf6f5/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.206135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca713fc-3462-4d26-8714-35dc8cdb4688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5e4f36d6a843b0804bfb72ffdee8b6, user_id=f6edb6c54e7f48dca958e09f3b6bf6f5, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:08.228083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dde165-d80c-4ef6-aaa5-6abc7d1e76e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:08.228083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde165-d80c-4ef6-aaa5-6abc7d1e76e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.228083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde165-d80c-4ef6-aaa5-6abc7d1e76e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.228083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dde165-d80c-4ef6-aaa5-6abc7d1e76e7 admin admin] PATH_INFO: `/v3/projects/09b04936e63a4f3fa923ef563857e4e0/users/48f60114d1124fa0a9b2d57c7f3e2923/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.228492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca713fc-3462-4d26-8714-35dc8cdb4688 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:08.228578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dde165-d80c-4ef6-aaa5-6abc7d1e76e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b04936e63a4f3fa923ef563857e4e0, user_id=48f60114d1124fa0a9b2d57c7f3e2923, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:08.228578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dde165-d80c-4ef6-aaa5-6abc7d1e76e7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:08.228578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:08.228814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 900/1783] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:08 2022] PUT /identity/v3/projects/dc5e4f36d6a843b0804bfb72ffdee8b6/users/f6edb6c54e7f48dca958e09f3b6bf6f5/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:08.228895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 884/1784] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:08 2022] PUT /identity/v3/projects/09b04936e63a4f3fa923ef563857e4e0/users/48f60114d1124fa0a9b2d57c7f3e2923/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:08.237320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5999f-c305-42d7-8a1e-b741d7e3422c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.237583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435aba3-8d50-4fd2-811a-92d7e842f016 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.237722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5999f-c305-42d7-8a1e-b741d7e3422c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.237899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5999f-c305-42d7-8a1e-b741d7e3422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.238000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435aba3-8d50-4fd2-811a-92d7e842f016 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.238152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435aba3-8d50-4fd2-811a-92d7e842f016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.267067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2bb5999f-c305-42d7-8a1e-b741d7e3422c None None] MFA Rules not processed for user `f6edb6c54e7f48dca958e09f3b6bf6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:08.270853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4435aba3-8d50-4fd2-811a-92d7e842f016 None None] MFA Rules not processed for user `48f60114d1124fa0a9b2d57c7f3e2923`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:08.270983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2bb5999f-c305-42d7-8a1e-b741d7e3422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:08.271083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4435aba3-8d50-4fd2-811a-92d7e842f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:08.365592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 901/1785] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:08.371272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 885/1786] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:08.385092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435aba3-8d50-4fd2-811a-92d7e842f016 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:08.386776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5999f-c305-42d7-8a1e-b741d7e3422c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:08.387099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4435aba3-8d50-4fd2-811a-92d7e842f016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:08.388185 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2bb5999f-c305-42d7-8a1e-b741d7e3422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:08.412366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:08.451752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435aba3-8d50-4fd2-811a-92d7e842f016 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:08.457541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4877c9-ac30-4d91-b3ed-0bdfe34cc64d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:08.458476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4877c9-ac30-4d91-b3ed-0bdfe34cc64d service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.458965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4877c9-ac30-4d91-b3ed-0bdfe34cc64d service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.459448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4877c9-ac30-4d91-b3ed-0bdfe34cc64d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.471106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4877c9-ac30-4d91-b3ed-0bdfe34cc64d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:08.472738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4877c9-ac30-4d91-b3ed-0bdfe34cc64d service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:08.477221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb5999f-c305-42d7-8a1e-b741d7e3422c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:08.481599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33da55-9b50-4b8f-bcb4-5249479780cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:08.482289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33da55-9b50-4b8f-bcb4-5249479780cd service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.482665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33da55-9b50-4b8f-bcb4-5249479780cd service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.482864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33da55-9b50-4b8f-bcb4-5249479780cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.485297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 886/1787] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:08.493454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33da55-9b50-4b8f-bcb4-5249479780cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:08.495017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33da55-9b50-4b8f-bcb4-5249479780cd service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:08.506009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 902/1788] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:08.788593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4877c9-ac30-4d91-b3ed-0bdfe34cc64d service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:08.860156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4877c9-ac30-4d91-b3ed-0bdfe34cc64d service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:08.865887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:08.868168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:08.868168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:08.868479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] PATH_INFO: `/v3/users/508e32cc95934fdf90c4ff1ae0fe393d` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:08.869711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508e32cc95934fdf90c4ff1ae0fe393d)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:08.871645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:08.992304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33da55-9b50-4b8f-bcb4-5249479780cd service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:09.001641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33da55-9b50-4b8f-bcb4-5249479780cd service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:09.005240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbe1ce-0f9c-4944-8721-b513d159afe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:09.005877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbe1ce-0f9c-4944-8721-b513d159afe4 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:09.006152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbe1ce-0f9c-4944-8721-b513d159afe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:09.006429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbe1ce-0f9c-4944-8721-b513d159afe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:09.062109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbe1ce-0f9c-4944-8721-b513d159afe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:09.063542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbe1ce-0f9c-4944-8721-b513d159afe4 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:09.072786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 903/1789] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:09.085771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508e32cc95934fdf90c4ff1ae0fe393d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:09.098855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:INSERT=1,keystone:SELECT=95,keystone:DELETE=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:09.179663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508e32cc95934fdf90c4ff1ae0fe393d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:09.180364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508e32cc95934fdf90c4ff1ae0fe393d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:09.187745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508e32cc95934fdf90c4ff1ae0fe393d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:09.188427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508e32cc95934fdf90c4ff1ae0fe393d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:09.208594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 887/1790] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:08 2022] DELETE /identity/v3/users/508e32cc95934fdf90c4ff1ae0fe393d => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:09.213868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b114ae-bd41-4f35-b985-1190f531c073 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:09.214163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b114ae-bd41-4f35-b985-1190f531c073 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:09.214444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b114ae-bd41-4f35-b985-1190f531c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:09.234183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c5b114ae-bd41-4f35-b985-1190f531c073 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:09.235364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c5b114ae-bd41-4f35-b985-1190f531c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:09.299658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 904/1791] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:09.309438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:09.315592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:09.378634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db560bb-1ab2-4ac7-8dc9-a83ba9ba4df6 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:09.382903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb7ce5-1dce-4a85-9458-fe61763fe0fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:09.384301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb7ce5-1dce-4a85-9458-fe61763fe0fe service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:09.384784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb7ce5-1dce-4a85-9458-fe61763fe0fe service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:09.385073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb7ce5-1dce-4a85-9458-fe61763fe0fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:09.394782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb7ce5-1dce-4a85-9458-fe61763fe0fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:09.396249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb7ce5-1dce-4a85-9458-fe61763fe0fe service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:09.407214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 888/1792] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:10.146462 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b114ae-bd41-4f35-b985-1190f531c073 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:10.153099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c5b114ae-bd41-4f35-b985-1190f531c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:10.169587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b114ae-bd41-4f35-b985-1190f531c073 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:10.176088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:10.177805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:10.178542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:10.178929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] PATH_INFO: `/v3/projects/f153234f1bd14f3b8f102795135d17ac` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:10.181070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f153234f1bd14f3b8f102795135d17ac)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:10.182941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:10.374285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f153234f1bd14f3b8f102795135d17ac', 'request_id': 'req-8ff20372-44ab-4e6d-8b39-22658efd9a30'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.442927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f153234f1bd14f3b8f102795135d17ac', 'request_id': 'req-8ff20372-44ab-4e6d-8b39-22658efd9a30'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.444484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:10.450025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f153234f1bd14f3b8f102795135d17ac', 'request_id': 'req-8ff20372-44ab-4e6d-8b39-22658efd9a30'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.462355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] The token cache is being invalidate because project f153234f1bd14f3b8f102795135d17ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:10.462648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.463311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.465019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 905/1793] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:10 2022] DELETE /identity/v3/projects/f153234f1bd14f3b8f102795135d17ac => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:10.469259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb7ce5-1dce-4a85-9458-fe61763fe0fe service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:10.479474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb7ce5-1dce-4a85-9458-fe61763fe0fe service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:10.556690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:10.557740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:10.558029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:10.558593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] PATH_INFO: `/v3/users/48f60114d1124fa0a9b2d57c7f3e2923` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:10.559687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f60114d1124fa0a9b2d57c7f3e2923)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:10.561846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:10.773548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:10.775672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f60114d1124fa0a9b2d57c7f3e2923'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.859975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f60114d1124fa0a9b2d57c7f3e2923'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.864461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f60114d1124fa0a9b2d57c7f3e2923'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.875529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f60114d1124fa0a9b2d57c7f3e2923'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.876396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4c745139-2ad4-4400-8e70-578267e14af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f60114d1124fa0a9b2d57c7f3e2923'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:10.912346 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 889/1794] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:10 2022] DELETE /identity/v3/users/48f60114d1124fa0a9b2d57c7f3e2923 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:11.552776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:11.554195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:11.569695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff20372-44ab-4e6d-8b39-22658efd9a30 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:11.627124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:11.627992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:11.628368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:11.632333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] PATH_INFO: `/v3/projects/09b04936e63a4f3fa923ef563857e4e0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:11.632333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b04936e63a4f3fa923ef563857e4e0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:11.632333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:11.762315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b04936e63a4f3fa923ef563857e4e0', 'request_id': 'req-56cc6581-a8ec-4a5b-aa92-c5902576416b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:11.877372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b04936e63a4f3fa923ef563857e4e0', 'request_id': 'req-56cc6581-a8ec-4a5b-aa92-c5902576416b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:11.886775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b04936e63a4f3fa923ef563857e4e0', 'request_id': 'req-56cc6581-a8ec-4a5b-aa92-c5902576416b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:11.901186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] The token cache is being invalidate because project 09b04936e63a4f3fa923ef563857e4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:11.901762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:11.902622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56cc6581-a8ec-4a5b-aa92-c5902576416b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:11.908162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 906/1795] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:11 2022] DELETE /identity/v3/projects/09b04936e63a4f3fa923ef563857e4e0 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:11.909649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d233183-16c6-43c9-8183-0b3d9a521b0d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:11.910092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d233183-16c6-43c9-8183-0b3d9a521b0d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:11.923245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d233183-16c6-43c9-8183-0b3d9a521b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:11.934512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2d233183-16c6-43c9-8183-0b3d9a521b0d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:11.935759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2d233183-16c6-43c9-8183-0b3d9a521b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:12.017652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 890/1796] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:12.026803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57299c42-4141-4309-849d-f2e102526a8b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.027122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57299c42-4141-4309-849d-f2e102526a8b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.027400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57299c42-4141-4309-849d-f2e102526a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.047257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-57299c42-4141-4309-849d-f2e102526a8b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:12.048622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-57299c42-4141-4309-849d-f2e102526a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:12.058458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 907/1797] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:12.063027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d233183-16c6-43c9-8183-0b3d9a521b0d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.073370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d233183-16c6-43c9-8183-0b3d9a521b0d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.078204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.079034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.079318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.079590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.080375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:12.080679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.082401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:12.096641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 891/1798] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:12.100767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57299c42-4141-4309-849d-f2e102526a8b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.111166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57299c42-4141-4309-849d-f2e102526a8b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.116315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a4616-2ecc-4f02-bfd8-00af5fded63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.116942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a4616-2ecc-4f02-bfd8-00af5fded63e admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.117354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a4616-2ecc-4f02-bfd8-00af5fded63e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.117848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a4616-2ecc-4f02-bfd8-00af5fded63e admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.118632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a4616-2ecc-4f02-bfd8-00af5fded63e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.120689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a4616-2ecc-4f02-bfd8-00af5fded63e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:12.246555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 908/1799] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:12 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:12.250049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.260842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee5edf-b54c-4219-875b-f40121721565 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.319381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f1bc-b516-4742-8a7b-e510d51c7f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.320057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f1bc-b516-4742-8a7b-e510d51c7f4f admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.320403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f1bc-b516-4742-8a7b-e510d51c7f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.320710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4952f1bc-b516-4742-8a7b-e510d51c7f4f admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.321403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952f1bc-b516-4742-8a7b-e510d51c7f4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.323302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4952f1bc-b516-4742-8a7b-e510d51c7f4f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:12.422150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 892/1800] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:12 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:12.423430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a4616-2ecc-4f02-bfd8-00af5fded63e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.433985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a4616-2ecc-4f02-bfd8-00af5fded63e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.438593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04ff77-b1c0-41ca-bf44-01c646658e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.439297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04ff77-b1c0-41ca-bf44-01c646658e8d admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.439579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04ff77-b1c0-41ca-bf44-01c646658e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.439906 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04ff77-b1c0-41ca-bf44-01c646658e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.440703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04ff77-b1c0-41ca-bf44-01c646658e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.442294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04ff77-b1c0-41ca-bf44-01c646658e8d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:12.454914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 909/1801] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:12.459858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f1bc-b516-4742-8a7b-e510d51c7f4f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.470596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4952f1bc-b516-4742-8a7b-e510d51c7f4f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.475318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021dec2-6096-4e4b-8caf-ecbb4cb2188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.476011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021dec2-6096-4e4b-8caf-ecbb4cb2188e admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.476329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021dec2-6096-4e4b-8caf-ecbb4cb2188e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.476664 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021dec2-6096-4e4b-8caf-ecbb4cb2188e admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.477407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021dec2-6096-4e4b-8caf-ecbb4cb2188e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.479008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7021dec2-6096-4e4b-8caf-ecbb4cb2188e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:12.489707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 893/1802] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:12.494169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04ff77-b1c0-41ca-bf44-01c646658e8d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.504389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04ff77-b1c0-41ca-bf44-01c646658e8d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.511607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcfe20-440f-42eb-aab0-c00f6304e1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.512547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcfe20-440f-42eb-aab0-c00f6304e1d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.512838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcfe20-440f-42eb-aab0-c00f6304e1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.513115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dcfe20-440f-42eb-aab0-c00f6304e1d9 admin admin] PATH_INFO: `/v3/projects/a2cb9765024f4fa4a90f68a7fbf7d408/users/d77256bea2964720aebf04416ff43824/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.537380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcfe20-440f-42eb-aab0-c00f6304e1d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2cb9765024f4fa4a90f68a7fbf7d408, user_id=d77256bea2964720aebf04416ff43824, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.537380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dcfe20-440f-42eb-aab0-c00f6304e1d9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:12.623219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 910/1803] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:12 2022] PUT /identity/v3/projects/a2cb9765024f4fa4a90f68a7fbf7d408/users/d77256bea2964720aebf04416ff43824/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:12.634243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641120ee-112d-413e-843f-740856f83d2f None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.635190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641120ee-112d-413e-843f-740856f83d2f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.635648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641120ee-112d-413e-843f-740856f83d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.666915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-641120ee-112d-413e-843f-740856f83d2f None None] MFA Rules not processed for user `d77256bea2964720aebf04416ff43824`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:12.668389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-641120ee-112d-413e-843f-740856f83d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:12.695833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:12.780326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 894/1804] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:12.782042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcfe20-440f-42eb-aab0-c00f6304e1d9 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.809439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:12.912143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dcfe20-440f-42eb-aab0-c00f6304e1d9 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.919294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97aeae-bb90-40f7-b67f-c04b343693b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.919953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97aeae-bb90-40f7-b67f-c04b343693b0 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.920281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97aeae-bb90-40f7-b67f-c04b343693b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.920565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97aeae-bb90-40f7-b67f-c04b343693b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.923184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641120ee-112d-413e-843f-740856f83d2f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:12.923669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97aeae-bb90-40f7-b67f-c04b343693b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.927009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-641120ee-112d-413e-843f-740856f83d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:12.927904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97aeae-bb90-40f7-b67f-c04b343693b0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:12.952330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641120ee-112d-413e-843f-740856f83d2f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:12.961282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:12.962241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:12.962564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:12.963644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] PATH_INFO: `/v3/users/b1e0a0d928c14ae295c296f5aa7d6d59` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:12.965617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e0a0d928c14ae295c296f5aa7d6d59)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:12.968027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:13.027451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 911/1805] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:12 2022] POST /identity/v3/projects => generated 379 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:13.033628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97aeae-bb90-40f7-b67f-c04b343693b0 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:13.207494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97aeae-bb90-40f7-b67f-c04b343693b0 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:13.221251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26c5dc-47ba-4409-af45-b3991ffb59de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:13.221960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26c5dc-47ba-4409-af45-b3991ffb59de admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.222236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26c5dc-47ba-4409-af45-b3991ffb59de admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.222519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26c5dc-47ba-4409-af45-b3991ffb59de admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.227242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26c5dc-47ba-4409-af45-b3991ffb59de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:13.227242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26c5dc-47ba-4409-af45-b3991ffb59de admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:13.273627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e0a0d928c14ae295c296f5aa7d6d59'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.304175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 912/1806] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:13 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:13.305981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26c5dc-47ba-4409-af45-b3991ffb59de admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:13.318344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26c5dc-47ba-4409-af45-b3991ffb59de admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:13.343627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e0a0d928c14ae295c296f5aa7d6d59'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.344304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e0a0d928c14ae295c296f5aa7d6d59'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.348016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:13.349516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.350136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.350512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.352318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:13.354182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:13.355745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e0a0d928c14ae295c296f5aa7d6d59'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.358740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a91f98fb-2c37-4884-bf1f-671d6e56b801 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e0a0d928c14ae295c296f5aa7d6d59'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:13.373113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 913/1807] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:13.378256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:13.381479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:13.411948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e4665-ca19-49a3-bc7b-faacb9fe1519 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:13.414724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 895/1808] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:12 2022] DELETE /identity/v3/users/b1e0a0d928c14ae295c296f5aa7d6d59 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:13.419605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98e05f-e01a-429f-a23d-deeff2fe6a9e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.419873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98e05f-e01a-429f-a23d-deeff2fe6a9e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.420178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98e05f-e01a-429f-a23d-deeff2fe6a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.462076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-5e98e05f-e01a-429f-a23d-deeff2fe6a9e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:13.463361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5e98e05f-e01a-429f-a23d-deeff2fe6a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:13.544592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:13.545838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.546398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.546888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.548148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:13.552181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:13.570331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 896/1809] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:13.579712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 914/1810] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:13.582202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98e05f-e01a-429f-a23d-deeff2fe6a9e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:13.582476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:13.585529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:13.602294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e98e05f-e01a-429f-a23d-deeff2fe6a9e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:13.608046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:13.619765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08dda0-3b7e-48e4-9f98-61e6304fced8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:13.621113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08dda0-3b7e-48e4-9f98-61e6304fced8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.621478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08dda0-3b7e-48e4-9f98-61e6304fced8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.622178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08dda0-3b7e-48e4-9f98-61e6304fced8 admin admin] PATH_INFO: `/v3/projects/73e5c8d29aa6475e9788d352da137448/users/1b3121051fad407881abbe53bad7744e/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.651644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08dda0-3b7e-48e4-9f98-61e6304fced8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e5c8d29aa6475e9788d352da137448, user_id=1b3121051fad407881abbe53bad7744e, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:13.654947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08dda0-3b7e-48e4-9f98-61e6304fced8 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:13.726115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec00829-00cd-4ea2-86bb-fe31cce9dfc0 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:13.733596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1440d2e-9580-45a5-9363-ff83dac6c6cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:13.734368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1440d2e-9580-45a5-9363-ff83dac6c6cd service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:13.735058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1440d2e-9580-45a5-9363-ff83dac6c6cd service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:13.735058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1440d2e-9580-45a5-9363-ff83dac6c6cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:13.749318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1440d2e-9580-45a5-9363-ff83dac6c6cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:13.787322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1440d2e-9580-45a5-9363-ff83dac6c6cd service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:13.806486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 915/1811] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:14.503592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 897/1812] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:13 2022] PUT /identity/v3/projects/73e5c8d29aa6475e9788d352da137448/users/1b3121051fad407881abbe53bad7744e/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 922 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:14.524243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428522e2-0195-4354-b73a-4852aab330ef None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.524497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428522e2-0195-4354-b73a-4852aab330ef None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.525832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428522e2-0195-4354-b73a-4852aab330ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.580159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-428522e2-0195-4354-b73a-4852aab330ef None None] MFA Rules not processed for user `1b3121051fad407881abbe53bad7744e`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:14.581655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-428522e2-0195-4354-b73a-4852aab330ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:14.685634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 916/1813] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:14.687090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08dda0-3b7e-48e4-9f98-61e6304fced8 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:14.706670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:14.836712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08dda0-3b7e-48e4-9f98-61e6304fced8 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:14.846561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:14.846561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:14.846561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:14.846561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:14.846905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:14.847245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:14.985936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 898/1814] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:14 2022] POST /identity/v3/projects => generated 377 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:14.993175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428522e2-0195-4354-b73a-4852aab330ef None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.026781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428522e2-0195-4354-b73a-4852aab330ef None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.106651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d01e4f-7c53-45f1-a5db-1a02b4ad30ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.107305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01e4f-7c53-45f1-a5db-1a02b4ad30ff admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.107592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01e4f-7c53-45f1-a5db-1a02b4ad30ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.107878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d01e4f-7c53-45f1-a5db-1a02b4ad30ff admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.108617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d01e4f-7c53-45f1-a5db-1a02b4ad30ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.110545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d01e4f-7c53-45f1-a5db-1a02b4ad30ff admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.189647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.191471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:15.205422 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1bbee-70ad-4efc-9988-fa00d90c916e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.211323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.212615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.212991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.214078 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] PATH_INFO: `/v3/projects/f672d88e89f1451d98118b5d696893bb` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.214808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f672d88e89f1451d98118b5d696893bb)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.217041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.242057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 917/1815] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:14 2022] POST /identity/v3/users => generated 295 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:15.243579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d01e4f-7c53-45f1-a5db-1a02b4ad30ff admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.260498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d01e4f-7c53-45f1-a5db-1a02b4ad30ff admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.267051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d8474-8b14-4098-8e05-26f1f397bf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.267908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d8474-8b14-4098-8e05-26f1f397bf42 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.268349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d8474-8b14-4098-8e05-26f1f397bf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.268731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d8474-8b14-4098-8e05-26f1f397bf42 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.269598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d8474-8b14-4098-8e05-26f1f397bf42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.272149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d8474-8b14-4098-8e05-26f1f397bf42 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.287320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 918/1816] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:15.292396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d8474-8b14-4098-8e05-26f1f397bf42 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.304343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d8474-8b14-4098-8e05-26f1f397bf42 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.310958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b359c5d-6351-4494-9be7-85ad9a4af951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.312094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b359c5d-6351-4494-9be7-85ad9a4af951 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.312343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b359c5d-6351-4494-9be7-85ad9a4af951 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.313240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b359c5d-6351-4494-9be7-85ad9a4af951 admin admin] PATH_INFO: `/v3/projects/224a1e38ecc74355b2fb2749b42c622e/users/c9b94998ff4446ed908522d966cf3477/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.330137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b359c5d-6351-4494-9be7-85ad9a4af951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224a1e38ecc74355b2fb2749b42c622e, user_id=c9b94998ff4446ed908522d966cf3477, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.332859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b359c5d-6351-4494-9be7-85ad9a4af951 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.392687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:15.401808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 919/1817] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:15 2022] PUT /identity/v3/projects/224a1e38ecc74355b2fb2749b42c622e/users/c9b94998ff4446ed908522d966cf3477/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:15.405588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b359c5d-6351-4494-9be7-85ad9a4af951 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.409159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f672d88e89f1451d98118b5d696893bb', 'request_id': 'req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.472716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b359c5d-6351-4494-9be7-85ad9a4af951 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.482757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f672d88e89f1451d98118b5d696893bb', 'request_id': 'req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.489635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f672d88e89f1451d98118b5d696893bb', 'request_id': 'req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.504861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] The token cache is being invalidate because project f672d88e89f1451d98118b5d696893bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:15.505585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.505983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.508707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 899/1818] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:15 2022] DELETE /identity/v3/projects/f672d88e89f1451d98118b5d696893bb => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:15.513022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.514702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:15.540220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.541234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.541529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.541816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.543237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.545103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.558010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 920/1819] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:15.561875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.563749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:15.578330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca8431-4d4b-42fa-82d9-65932a0270b8 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.583021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452d13c-0988-48f2-9b05-2f30b46e58d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.583980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452d13c-0988-48f2-9b05-2f30b46e58d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.584309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452d13c-0988-48f2-9b05-2f30b46e58d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.584586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452d13c-0988-48f2-9b05-2f30b46e58d4 admin admin] PATH_INFO: `/v3/projects/224a1e38ecc74355b2fb2749b42c622e/users/c9b94998ff4446ed908522d966cf3477/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.587628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452d13c-0988-48f2-9b05-2f30b46e58d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=224a1e38ecc74355b2fb2749b42c622e, user_id=c9b94998ff4446ed908522d966cf3477, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.590674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452d13c-0988-48f2-9b05-2f30b46e58d4 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.597016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7f75d-f06b-40ae-b756-3bb6a6f28629 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.603876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 921/1820] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:15 2022] PUT /identity/v3/projects/224a1e38ecc74355b2fb2749b42c622e/users/c9b94998ff4446ed908522d966cf3477/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:15.617763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475358-a21e-4b39-9f47-3b2db3af822c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.618128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475358-a21e-4b39-9f47-3b2db3af822c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.618568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90475358-a21e-4b39-9f47-3b2db3af822c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.646784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-90475358-a21e-4b39-9f47-3b2db3af822c None None] MFA Rules not processed for user `c9b94998ff4446ed908522d966cf3477`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:15.649264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-90475358-a21e-4b39-9f47-3b2db3af822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:15.662601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.664929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.664929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.664929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] PATH_INFO: `/v3/users/f6edb6c54e7f48dca958e09f3b6bf6f5` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.665585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6edb6c54e7f48dca958e09f3b6bf6f5)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.667511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.766955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 922/1821] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:15.780044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90475358-a21e-4b39-9f47-3b2db3af822c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.784342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-90475358-a21e-4b39-9f47-3b2db3af822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:15.874697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90475358-a21e-4b39-9f47-3b2db3af822c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:15.878618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:15.879299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:15.879612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:15.880020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:15.890435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:15.891878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:15.905864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:15.906482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 923/1822] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:15.929870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6edb6c54e7f48dca958e09f3b6bf6f5'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.974055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6edb6c54e7f48dca958e09f3b6bf6f5'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.974888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6edb6c54e7f48dca958e09f3b6bf6f5'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.987841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6edb6c54e7f48dca958e09f3b6bf6f5'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.988528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6edb6c54e7f48dca958e09f3b6bf6f5'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:15.995689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:15.997796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:16.009889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 900/1823] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:15 2022] DELETE /identity/v3/users/f6edb6c54e7f48dca958e09f3b6bf6f5 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:16.010911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d349f-b35d-4596-8b05-32a595dad5f2 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:16.054608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:16.089148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc0b130-9a60-4a96-b9ac-9cf8fe74db4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:16.089148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0b130-9a60-4a96-b9ac-9cf8fe74db4f service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.089148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0b130-9a60-4a96-b9ac-9cf8fe74db4f service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.089148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc0b130-9a60-4a96-b9ac-9cf8fe74db4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.098322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6dc0b130-9a60-4a96-b9ac-9cf8fe74db4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:16.181142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0b130-9a60-4a96-b9ac-9cf8fe74db4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:16.182625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc0b130-9a60-4a96-b9ac-9cf8fe74db4f service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.192937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 924/1824] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:16.638704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:16.640309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:16.727870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed523dab-c3c5-4346-bff0-9e5f19b52f8a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:16.732706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:16.733581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:16.733863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:16.734137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] PATH_INFO: `/v3/projects/dc5e4f36d6a843b0804bfb72ffdee8b6` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:16.735328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5e4f36d6a843b0804bfb72ffdee8b6)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:16.737290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:16.890850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:16.897424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5e4f36d6a843b0804bfb72ffdee8b6', 'request_id': 'req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:16.979842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5e4f36d6a843b0804bfb72ffdee8b6', 'request_id': 'req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:16.987506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5e4f36d6a843b0804bfb72ffdee8b6', 'request_id': 'req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:17.008000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] The token cache is being invalidate because project dc5e4f36d6a843b0804bfb72ffdee8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:17.008890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:17.009756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0a4a7a1c-8451-4ee0-a8c1-e560f30232d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:17.011728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 901/1825] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:16 2022] DELETE /identity/v3/projects/dc5e4f36d6a843b0804bfb72ffdee8b6 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:17.017562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df570589-b612-4b3c-b54f-748fb82af5d5 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.019438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df570589-b612-4b3c-b54f-748fb82af5d5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.020012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df570589-b612-4b3c-b54f-748fb82af5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.042125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-df570589-b612-4b3c-b54f-748fb82af5d5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:17.043746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-df570589-b612-4b3c-b54f-748fb82af5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:17.154512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 925/1826] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:17.164342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5840b0-3872-46dd-ab39-440529f4f321 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.164342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5840b0-3872-46dd-ab39-440529f4f321 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.164342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5840b0-3872-46dd-ab39-440529f4f321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.197628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1e5840b0-3872-46dd-ab39-440529f4f321 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:17.199030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1e5840b0-3872-46dd-ab39-440529f4f321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:17.207905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 902/1827] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:17.213839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df570589-b612-4b3c-b54f-748fb82af5d5 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.226961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df570589-b612-4b3c-b54f-748fb82af5d5 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.238197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.239274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.239274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.239583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.240393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:17.243023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.245496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.255126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:17.275642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 926/1828] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:17.281984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5840b0-3872-46dd-ab39-440529f4f321 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.307991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5840b0-3872-46dd-ab39-440529f4f321 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.317256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f4327-272f-49fc-a147-48ab53a67b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.318480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f4327-272f-49fc-a147-48ab53a67b5d admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.319085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f4327-272f-49fc-a147-48ab53a67b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.319373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f4327-272f-49fc-a147-48ab53a67b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.320253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f4327-272f-49fc-a147-48ab53a67b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.322528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f4327-272f-49fc-a147-48ab53a67b5d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.455531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 903/1829] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:17 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:17.459799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.482801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cbda01-ad61-4636-92c3-976763e0eaa7 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.542015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30934c09-e719-4a21-bdc7-cbf95f4c2b44 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.542382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30934c09-e719-4a21-bdc7-cbf95f4c2b44 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.542667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30934c09-e719-4a21-bdc7-cbf95f4c2b44 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.545408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 904/1830] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 06:00:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:00:17.559489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1fda9-cd60-4aaf-9501-0a2564b087e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.560457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1fda9-cd60-4aaf-9501-0a2564b087e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.560747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1fda9-cd60-4aaf-9501-0a2564b087e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.564181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1fda9-cd60-4aaf-9501-0a2564b087e5 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.564181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1fda9-cd60-4aaf-9501-0a2564b087e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.564460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37856a5a-2ac1-4feb-9069-4cb86107c10d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.564460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37856a5a-2ac1-4feb-9069-4cb86107c10d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.564460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37856a5a-2ac1-4feb-9069-4cb86107c10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.564668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d1fda9-cd60-4aaf-9501-0a2564b087e5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.592881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-37856a5a-2ac1-4feb-9069-4cb86107c10d None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:17.594520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-37856a5a-2ac1-4feb-9069-4cb86107c10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:17.672231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 927/1831] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:17 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:17.673474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1fda9-cd60-4aaf-9501-0a2564b087e5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.686570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1fda9-cd60-4aaf-9501-0a2564b087e5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.691108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58964d16-8be1-4947-9dd9-b65917516037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.691847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58964d16-8be1-4947-9dd9-b65917516037 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.692100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58964d16-8be1-4947-9dd9-b65917516037 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.693833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58964d16-8be1-4947-9dd9-b65917516037 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.694141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 905/1832] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 06:00:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:17.694572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58964d16-8be1-4947-9dd9-b65917516037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.694846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58964d16-8be1-4947-9dd9-b65917516037 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.707778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37856a5a-2ac1-4feb-9069-4cb86107c10d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.709944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-37856a5a-2ac1-4feb-9069-4cb86107c10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:17.715388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 928/1833] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:17.716370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58964d16-8be1-4947-9dd9-b65917516037 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.728236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37856a5a-2ac1-4feb-9069-4cb86107c10d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.734234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58964d16-8be1-4947-9dd9-b65917516037 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.740760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e20d1-0a11-4cdb-92ea-05e0264e3827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.741432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e20d1-0a11-4cdb-92ea-05e0264e3827 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.741716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e20d1-0a11-4cdb-92ea-05e0264e3827 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.742680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e20d1-0a11-4cdb-92ea-05e0264e3827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.753702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e20d1-0a11-4cdb-92ea-05e0264e3827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.755053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e20d1-0a11-4cdb-92ea-05e0264e3827 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.756980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ac8e2-7443-4f79-ac78-6258f36a5dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.757331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac8e2-7443-4f79-ac78-6258f36a5dce admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.757978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac8e2-7443-4f79-ac78-6258f36a5dce admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.758268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ac8e2-7443-4f79-ac78-6258f36a5dce admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.759025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac8e2-7443-4f79-ac78-6258f36a5dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.760685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ac8e2-7443-4f79-ac78-6258f36a5dce admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.771755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 906/1834] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:17.780777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 929/1835] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:17.781720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e20d1-0a11-4cdb-92ea-05e0264e3827 service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:17.793714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e20d1-0a11-4cdb-92ea-05e0264e3827 service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:17.799321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8303dcc-5591-4813-9b2d-8bab8706d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:17.800258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303dcc-5591-4813-9b2d-8bab8706d459 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.800545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303dcc-5591-4813-9b2d-8bab8706d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.800815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303dcc-5591-4813-9b2d-8bab8706d459 admin admin] PATH_INFO: `/v3/projects/ef40fb8af8f7460b8eb9132e3b17d241/users/04296ac6669e4c149f6bfeaae925dd60/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.817685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8303dcc-5591-4813-9b2d-8bab8706d459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef40fb8af8f7460b8eb9132e3b17d241, user_id=04296ac6669e4c149f6bfeaae925dd60, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:17.820724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8303dcc-5591-4813-9b2d-8bab8706d459 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:17.946237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 907/1836] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:17 2022] PUT /identity/v3/projects/ef40fb8af8f7460b8eb9132e3b17d241/users/04296ac6669e4c149f6bfeaae925dd60/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:17.957902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851627-8823-4713-af39-5e5aad3ae113 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:17.958196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851627-8823-4713-af39-5e5aad3ae113 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:17.958560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d851627-8823-4713-af39-5e5aad3ae113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:17.987182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1d851627-8823-4713-af39-5e5aad3ae113 None None] MFA Rules not processed for user `04296ac6669e4c149f6bfeaae925dd60`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:17.988451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1d851627-8823-4713-af39-5e5aad3ae113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:18.065062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 930/1837] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:18.069659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8303dcc-5591-4813-9b2d-8bab8706d459 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:18.138023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:18.143683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8303dcc-5591-4813-9b2d-8bab8706d459 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:18.150482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbae57-919c-4556-9a3d-834991a2a42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:18.151253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbae57-919c-4556-9a3d-834991a2a42b admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:18.151542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbae57-919c-4556-9a3d-834991a2a42b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:18.157339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbae57-919c-4556-9a3d-834991a2a42b admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:18.157339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbae57-919c-4556-9a3d-834991a2a42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:18.157339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbae57-919c-4556-9a3d-834991a2a42b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:18.286424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 908/1838] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:18 2022] POST /identity/v3/projects => generated 379 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:18.291235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851627-8823-4713-af39-5e5aad3ae113 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:18.307798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d851627-8823-4713-af39-5e5aad3ae113 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:18.435808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:18.444298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676345-cc2d-4ea4-bbce-78bf965769fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:18.445207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676345-cc2d-4ea4-bbce-78bf965769fc admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:18.445570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676345-cc2d-4ea4-bbce-78bf965769fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:18.446083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7676345-cc2d-4ea4-bbce-78bf965769fc admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:18.447058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7676345-cc2d-4ea4-bbce-78bf965769fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:18.449290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7676345-cc2d-4ea4-bbce-78bf965769fc admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:18.556244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 931/1839] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:18 2022] POST /identity/v3/users => generated 296 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:18.560102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbae57-919c-4556-9a3d-834991a2a42b admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:18.575357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbae57-919c-4556-9a3d-834991a2a42b admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:18.583882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09dd16-1889-4e23-b414-4f109ab1a8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:18.584638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09dd16-1889-4e23-b414-4f109ab1a8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:18.584920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09dd16-1889-4e23-b414-4f109ab1a8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:18.585197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09dd16-1889-4e23-b414-4f109ab1a8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:18.586106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09dd16-1889-4e23-b414-4f109ab1a8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:18.587716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09dd16-1889-4e23-b414-4f109ab1a8e0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:18.600200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 909/1840] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:18.606597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676345-cc2d-4ea4-bbce-78bf965769fc admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:18.617555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7676345-cc2d-4ea4-bbce-78bf965769fc admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:18.622793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b50b-bcf8-46c7-bd00-cd40178dfec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:18.623697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b50b-bcf8-46c7-bd00-cd40178dfec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:18.623979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b50b-bcf8-46c7-bd00-cd40178dfec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:18.624311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca1b50b-bcf8-46c7-bd00-cd40178dfec6 admin admin] PATH_INFO: `/v3/projects/ef2f87fec01a488b976835e0f5965b89/users/acc5b11711fc437e8fde3ecc453e3853/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:18.642282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1b50b-bcf8-46c7-bd00-cd40178dfec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2f87fec01a488b976835e0f5965b89, user_id=acc5b11711fc437e8fde3ecc453e3853, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:18.645664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca1b50b-bcf8-46c7-bd00-cd40178dfec6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:18.725206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 932/1841] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:18 2022] PUT /identity/v3/projects/ef2f87fec01a488b976835e0f5965b89/users/acc5b11711fc437e8fde3ecc453e3853/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:18.731054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09dd16-1889-4e23-b414-4f109ab1a8e0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:18.746308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09dd16-1889-4e23-b414-4f109ab1a8e0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:18.849185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6078c2-6f22-41b0-9482-2d8470b9be56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:18.849891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6078c2-6f22-41b0-9482-2d8470b9be56 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:18.862378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6078c2-6f22-41b0-9482-2d8470b9be56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:18.862378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6078c2-6f22-41b0-9482-2d8470b9be56 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:18.862378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6078c2-6f22-41b0-9482-2d8470b9be56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:18.862378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6078c2-6f22-41b0-9482-2d8470b9be56 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:18.864272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 910/1842] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:18.868734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b50b-bcf8-46c7-bd00-cd40178dfec6 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:18.878952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca1b50b-bcf8-46c7-bd00-cd40178dfec6 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:18.883768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:18.884783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:18.885134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:18.885479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] PATH_INFO: `/v3/projects/ef2f87fec01a488b976835e0f5965b89/users/acc5b11711fc437e8fde3ecc453e3853/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:18.887596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2f87fec01a488b976835e0f5965b89, user_id=acc5b11711fc437e8fde3ecc453e3853, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:18.890313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:18.901751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 933/1843] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:18 2022] PUT /identity/v3/projects/ef2f87fec01a488b976835e0f5965b89/users/acc5b11711fc437e8fde3ecc453e3853/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:18.911897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab00cd-2e7a-4167-8ccf-135bb363ea0b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:18.912224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab00cd-2e7a-4167-8ccf-135bb363ea0b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:18.912512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bab00cd-2e7a-4167-8ccf-135bb363ea0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:18.939675 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9bab00cd-2e7a-4167-8ccf-135bb363ea0b None None] MFA Rules not processed for user `acc5b11711fc437e8fde3ecc453e3853`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:18.940965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9bab00cd-2e7a-4167-8ccf-135bb363ea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:19.032914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 911/1844] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:19.046570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:19.048692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:19.116171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad5de7-35ca-4e56-8115-9ff455bf955b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:19.120991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85c31-2e79-4376-9914-de252d0cb93a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:19.121669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85c31-2e79-4376-9914-de252d0cb93a service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:19.121953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85c31-2e79-4376-9914-de252d0cb93a service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:19.123064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85c31-2e79-4376-9914-de252d0cb93a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:19.131951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e85c31-2e79-4376-9914-de252d0cb93a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:19.134077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e85c31-2e79-4376-9914-de252d0cb93a service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:19.144007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 934/1845] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:20.230126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab00cd-2e7a-4167-8ccf-135bb363ea0b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:20.243022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bab00cd-2e7a-4167-8ccf-135bb363ea0b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:20.248686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc5b11711fc437e8fde3ecc453e3853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2f87fec01a488b976835e0f5965b89', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:20.249482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:20.250051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:20.250399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:20.251157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:20.251506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:20.253184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:20.265935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 912/1846] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:20.270089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85c31-2e79-4376-9914-de252d0cb93a service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:20.279608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85c31-2e79-4376-9914-de252d0cb93a service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:20.284220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a4a8-da73-4da5-bfec-35e401b3f7b4 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc5b11711fc437e8fde3ecc453e3853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2f87fec01a488b976835e0f5965b89', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:20.284857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a4a8-da73-4da5-bfec-35e401b3f7b4 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:20.285147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a4a8-da73-4da5-bfec-35e401b3f7b4 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:20.285431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3795a4a8-da73-4da5-bfec-35e401b3f7b4 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:20.286126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795a4a8-da73-4da5-bfec-35e401b3f7b4 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:20.288090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3795a4a8-da73-4da5-bfec-35e401b3f7b4 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:20.392829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 935/1847] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:20 2022] POST /identity/v3/projects => generated 355 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:20.396838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:20.407875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6530b61-33be-4488-9b4e-ed99a26ad3c7 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:20.453129 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:20.474344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc5b11711fc437e8fde3ecc453e3853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2f87fec01a488b976835e0f5965b89', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:20.475098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:20.475387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:20.475669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:20.476676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:20.477186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:20.478804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:20.490264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 913/1848] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:20.558742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a4a8-da73-4da5-bfec-35e401b3f7b4 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:20.579585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3795a4a8-da73-4da5-bfec-35e401b3f7b4 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:20.580360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:20.582233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:20.587663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'acc5b11711fc437e8fde3ecc453e3853', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef2f87fec01a488b976835e0f5965b89', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:20.588922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:20.589746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:20.590250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] PATH_INFO: `/v3/projects/7a369a0412a34328af403e20d9b20273` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:20.591889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7a369a0412a34328af403e20d9b20273)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:20.594266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:20.690052 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2d4c4-0b6c-43a0-a9c0-c6ab75f4b08f tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:20.700064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:20.702578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:20.703352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:20.704198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] PATH_INFO: `/v3/users/d77256bea2964720aebf04416ff43824` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:20.706100 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d77256bea2964720aebf04416ff43824)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:20.708431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:20.747697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a369a0412a34328af403e20d9b20273', 'request_id': 'req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.831087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a369a0412a34328af403e20d9b20273', 'request_id': 'req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.837921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a369a0412a34328af403e20d9b20273', 'request_id': 'req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.849174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] The token cache is being invalidate because project 7a369a0412a34328af403e20d9b20273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:20.849461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.850106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1662d6fb-4e34-4d8d-ace1-bd428dcb6855 tempest-VolumeQuotaClassesTest-753150913 tempest-VolumeQuotaClassesTest-753150913-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:20.852337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 936/1849] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:20 2022] DELETE /identity/v3/projects/7a369a0412a34328af403e20d9b20273 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:20.936804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd77256bea2964720aebf04416ff43824'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.002186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd77256bea2964720aebf04416ff43824'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.002872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd77256bea2964720aebf04416ff43824'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.010550 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd77256bea2964720aebf04416ff43824'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.011194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd77256bea2964720aebf04416ff43824'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.029772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 914/1850] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:20 2022] DELETE /identity/v3/users/d77256bea2964720aebf04416ff43824 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:21.034282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd5d6-0586-46b5-8c7a-33882f5a67bd None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.034574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd5d6-0586-46b5-8c7a-33882f5a67bd None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.034856 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abd5d6-0586-46b5-8c7a-33882f5a67bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.053007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-a6abd5d6-0586-46b5-8c7a-33882f5a67bd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:21.054256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a6abd5d6-0586-46b5-8c7a-33882f5a67bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:21.059490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:21.114516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 937/1851] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:21.124306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.125755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:21.191415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd73be4-beab-4890-9c2b-4b10af35f911 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.195704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.196470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.196763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.197044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.204744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:21.207415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.209084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.219822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 915/1852] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:21.548718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abd5d6-0586-46b5-8c7a-33882f5a67bd None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.550181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a6abd5d6-0586-46b5-8c7a-33882f5a67bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:21.561966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abd5d6-0586-46b5-8c7a-33882f5a67bd None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.566745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.567625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.567903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.568214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] PATH_INFO: `/v3/users/04296ac6669e4c149f6bfeaae925dd60` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.571112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04296ac6669e4c149f6bfeaae925dd60)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.571854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.788892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04296ac6669e4c149f6bfeaae925dd60'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.809951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:21.809951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:21.832005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5f7eb-5deb-4754-8bd6-60459a774fa4 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:21.840965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:21.840965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.840965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.840965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] PATH_INFO: `/v3/projects/a2cb9765024f4fa4a90f68a7fbf7d408` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.853868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2cb9765024f4fa4a90f68a7fbf7d408)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:21.860249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:21.890558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04296ac6669e4c149f6bfeaae925dd60'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.891691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04296ac6669e4c149f6bfeaae925dd60'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.912780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04296ac6669e4c149f6bfeaae925dd60'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.914048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-621aa2be-65c4-4de7-9501-ce7390dee453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04296ac6669e4c149f6bfeaae925dd60'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:21.941055 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 938/1853] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:21 2022] DELETE /identity/v3/users/04296ac6669e4c149f6bfeaae925dd60 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:21.942091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9541-9355-47dc-8676-982a340c8e8b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:21.942389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9541-9355-47dc-8676-982a340c8e8b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:21.942672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a9541-9355-47dc-8676-982a340c8e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:21.969652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-bb2a9541-9355-47dc-8676-982a340c8e8b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:21.969652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bb2a9541-9355-47dc-8676-982a340c8e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:22.014707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2cb9765024f4fa4a90f68a7fbf7d408', 'request_id': 'req-08a75c7b-d7be-45bc-9616-43446c454bd3'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.074814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 939/1854] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:22.081355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a9541-9355-47dc-8676-982a340c8e8b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:22.082943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bb2a9541-9355-47dc-8676-982a340c8e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:22.143908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2cb9765024f4fa4a90f68a7fbf7d408', 'request_id': 'req-08a75c7b-d7be-45bc-9616-43446c454bd3'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.144382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a9541-9355-47dc-8676-982a340c8e8b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:22.151980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2cb9765024f4fa4a90f68a7fbf7d408', 'request_id': 'req-08a75c7b-d7be-45bc-9616-43446c454bd3'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.168342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:22.175238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] The token cache is being invalidate because project a2cb9765024f4fa4a90f68a7fbf7d408 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:22.175525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.176776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.184911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 916/1855] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:21 2022] DELETE /identity/v3/projects/a2cb9765024f4fa4a90f68a7fbf7d408 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:22.185916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:22.190496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:22.216244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:22.216244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:22.216695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:22.217044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:22.218973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:22.281177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a75c7b-d7be-45bc-9616-43446c454bd3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:22.287849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:22.289308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:22.289656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:22.289869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] PATH_INFO: `/v3/users/1b3121051fad407881abbe53bad7744e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:22.291245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3121051fad407881abbe53bad7744e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:22.294608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:22.297994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:22.299743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:22.311537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 940/1856] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:22.484875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3121051fad407881abbe53bad7744e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.583732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3121051fad407881abbe53bad7744e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.584034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3121051fad407881abbe53bad7744e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.595047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3121051fad407881abbe53bad7744e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.595768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3121051fad407881abbe53bad7744e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:22.610805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:22.615378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 917/1857] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:22 2022] DELETE /identity/v3/users/1b3121051fad407881abbe53bad7744e => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:22.853974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:22.855478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:22.867741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f809ee-d366-4bf5-9947-96311350d997 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:22.943028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:22.943912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:22.944234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:22.944516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] PATH_INFO: `/v3/projects/ef40fb8af8f7460b8eb9132e3b17d241` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:22.946278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef40fb8af8f7460b8eb9132e3b17d241)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:22.948406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:23.128935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef40fb8af8f7460b8eb9132e3b17d241', 'request_id': 'req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.143597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:23.145075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:23.179054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152449c-4032-4f64-aaae-2dc63868697b admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:23.251263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef40fb8af8f7460b8eb9132e3b17d241', 'request_id': 'req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.257866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef40fb8af8f7460b8eb9132e3b17d241', 'request_id': 'req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.263154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:23.264168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:23.264488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:23.264767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] PATH_INFO: `/v3/projects/73e5c8d29aa6475e9788d352da137448` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:23.266000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e5c8d29aa6475e9788d352da137448)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:23.267923 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:23.272497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] The token cache is being invalidate because project ef40fb8af8f7460b8eb9132e3b17d241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:23.272793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.273454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.276592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 941/1858] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:22 2022] DELETE /identity/v3/projects/ef40fb8af8f7460b8eb9132e3b17d241 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:23.279285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:23.281545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:23.284931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:23.299236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc7f3e-f20a-44ef-8371-7d54453d78cf admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:23.304371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:23.305918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:23.306228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:23.306511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] PATH_INFO: `/v3/users/acc5b11711fc437e8fde3ecc453e3853` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:23.307765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acc5b11711fc437e8fde3ecc453e3853)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:23.309811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:23.412566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e5c8d29aa6475e9788d352da137448', 'request_id': 'req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.494000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e5c8d29aa6475e9788d352da137448', 'request_id': 'req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.500564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e5c8d29aa6475e9788d352da137448', 'request_id': 'req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.518026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] The token cache is being invalidate because project 73e5c8d29aa6475e9788d352da137448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:23.518566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.518715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acc5b11711fc437e8fde3ecc453e3853'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.519701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.523436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 918/1859] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:23 2022] DELETE /identity/v3/projects/73e5c8d29aa6475e9788d352da137448 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:23.529442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:23.533613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:23.598946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7fb15d-7559-4abb-9a9b-5bce75cfefe0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:23.605715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:23.606530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:23.613451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:23.613451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] PATH_INFO: `/v3/users/c9b94998ff4446ed908522d966cf3477` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:23.613451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b94998ff4446ed908522d966cf3477)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:23.613451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:23.635398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acc5b11711fc437e8fde3ecc453e3853'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.636149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acc5b11711fc437e8fde3ecc453e3853'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.645908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acc5b11711fc437e8fde3ecc453e3853'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.646663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acc5b11711fc437e8fde3ecc453e3853'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.670490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 942/1860] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:23 2022] DELETE /identity/v3/users/acc5b11711fc437e8fde3ecc453e3853 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:23.801934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b94998ff4446ed908522d966cf3477'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.911721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b94998ff4446ed908522d966cf3477'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.912547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b94998ff4446ed908522d966cf3477'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.921258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b94998ff4446ed908522d966cf3477'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.922969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b94998ff4446ed908522d966cf3477'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:23.942753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 919/1861] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:23 2022] DELETE /identity/v3/users/c9b94998ff4446ed908522d966cf3477 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:23.968392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:24.193188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:24.194706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:24.298600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55248c3-49e1-48d2-a440-95aa65b94245 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:24.349540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:24.379205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:24.380164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:24.380448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:24.380735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] PATH_INFO: `/v3/projects/ef2f87fec01a488b976835e0f5965b89` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:24.381969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2f87fec01a488b976835e0f5965b89)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:24.384768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:24.488548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2f87fec01a488b976835e0f5965b89', 'request_id': 'req-5dfd0684-c33b-4138-9cd7-2257c4705137'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.561134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2f87fec01a488b976835e0f5965b89', 'request_id': 'req-5dfd0684-c33b-4138-9cd7-2257c4705137'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.568464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2f87fec01a488b976835e0f5965b89', 'request_id': 'req-5dfd0684-c33b-4138-9cd7-2257c4705137'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.577332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:24.578770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:24.582410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] The token cache is being invalidate because project ef2f87fec01a488b976835e0f5965b89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:24.582804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.583535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5dfd0684-c33b-4138-9cd7-2257c4705137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.589400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 943/1862] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:24 2022] DELETE /identity/v3/projects/ef2f87fec01a488b976835e0f5965b89 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:24.592155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb8f75-4c52-412d-8b4d-0ad67f7f3a08 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:24.593655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb8f75-4c52-412d-8b4d-0ad67f7f3a08 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:24.593655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb8f75-4c52-412d-8b4d-0ad67f7f3a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:24.594624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c842e13a-ff15-4a4c-a35d-045d45b9fa47 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:24.619526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-05bb8f75-4c52-412d-8b4d-0ad67f7f3a08 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:24.621771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-05bb8f75-4c52-412d-8b4d-0ad67f7f3a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:24.733251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 944/1863] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:24.736611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:24.737978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:24.738438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:24.738939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] PATH_INFO: `/v3/projects/224a1e38ecc74355b2fb2749b42c622e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:24.740462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=224a1e38ecc74355b2fb2749b42c622e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:24.741214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e0fa-fb15-4c7d-91ce-32524562ef89 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:24.741534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e0fa-fb15-4c7d-91ce-32524562ef89 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:24.741815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145e0fa-fb15-4c7d-91ce-32524562ef89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:24.742694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:24.777109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d145e0fa-fb15-4c7d-91ce-32524562ef89 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:24.778366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d145e0fa-fb15-4c7d-91ce-32524562ef89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:24.789823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 945/1864] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:24.793015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e0fa-fb15-4c7d-91ce-32524562ef89 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:24.803151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145e0fa-fb15-4c7d-91ce-32524562ef89 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:24.808266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:24.808997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:24.809279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:24.809553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:24.810289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:24.810594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:24.812540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:24.831165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 946/1865] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:24.832206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:24.842635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f9722-ed5e-45f2-a236-eccaaadb8345 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:24.847461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a43c2-154f-407b-8ca8-8d395a7ec28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:24.848130 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a43c2-154f-407b-8ca8-8d395a7ec28f admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:24.848433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a43c2-154f-407b-8ca8-8d395a7ec28f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:24.848718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a43c2-154f-407b-8ca8-8d395a7ec28f admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:24.849415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a43c2-154f-407b-8ca8-8d395a7ec28f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:24.851343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a43c2-154f-407b-8ca8-8d395a7ec28f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:24.876267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '224a1e38ecc74355b2fb2749b42c622e', 'request_id': 'req-845b169a-f921-43f4-842f-7ae06d6e8327'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.935651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 947/1866] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:24 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:24.939789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a43c2-154f-407b-8ca8-8d395a7ec28f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:24.956415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '224a1e38ecc74355b2fb2749b42c622e', 'request_id': 'req-845b169a-f921-43f4-842f-7ae06d6e8327'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.963142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '224a1e38ecc74355b2fb2749b42c622e', 'request_id': 'req-845b169a-f921-43f4-842f-7ae06d6e8327'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.976198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] The token cache is being invalidate because project 224a1e38ecc74355b2fb2749b42c622e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:24.976762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.977719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-845b169a-f921-43f4-842f-7ae06d6e8327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:24.979673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 920/1867] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:24 2022] DELETE /identity/v3/projects/224a1e38ecc74355b2fb2749b42c622e => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:24.986017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a84b-27eb-472a-8e00-541ba4c9318a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:24.986312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a84b-27eb-472a-8e00-541ba4c9318a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:24.986989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00a84b-27eb-472a-8e00-541ba4c9318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.007969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a43c2-154f-407b-8ca8-8d395a7ec28f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.010023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-5d00a84b-27eb-472a-8e00-541ba4c9318a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:25.011464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5d00a84b-27eb-472a-8e00-541ba4c9318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.013854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.014554 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.014998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.015286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.015998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.018621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.027682 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 921/1868] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:25.031398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f8384-2ae8-4d98-ac6a-0b064cfe956c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.031682 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f8384-2ae8-4d98-ac6a-0b064cfe956c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.031953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f8384-2ae8-4d98-ac6a-0b064cfe956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.052316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-446f8384-2ae8-4d98-ac6a-0b064cfe956c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:25.053544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-446f8384-2ae8-4d98-ac6a-0b064cfe956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.062568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 922/1869] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:25.066854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f8384-2ae8-4d98-ac6a-0b064cfe956c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.077576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f8384-2ae8-4d98-ac6a-0b064cfe956c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.083073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.083777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.084050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.084359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.085101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:25.085407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.087136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.104770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 923/1870] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:25.105962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.117102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d058b-33b2-476d-881f-a1fbdf966a59 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.123257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022af046-a336-4159-b0b4-437085000dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.123935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af046-a336-4159-b0b4-437085000dfc admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.127944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af046-a336-4159-b0b4-437085000dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.127944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022af046-a336-4159-b0b4-437085000dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.128151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 948/1871] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:24 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:25.128236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022af046-a336-4159-b0b4-437085000dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.128236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022af046-a336-4159-b0b4-437085000dfc admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.129156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.131114 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.140816 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:25.147462 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91604086-39df-4a96-9a27-159adb682a79 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.155730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982951c-3763-4924-8758-eaf8ba56474b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.156553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982951c-3763-4924-8758-eaf8ba56474b admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.156912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982951c-3763-4924-8758-eaf8ba56474b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.157271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982951c-3763-4924-8758-eaf8ba56474b admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.158194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982951c-3763-4924-8758-eaf8ba56474b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.159907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e982951c-3763-4924-8758-eaf8ba56474b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.176375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 949/1872] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:25.178961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982951c-3763-4924-8758-eaf8ba56474b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.192396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e982951c-3763-4924-8758-eaf8ba56474b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.200960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f86b2c-a7a3-4463-b2c3-a8fb0de61f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.202072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f86b2c-a7a3-4463-b2c3-a8fb0de61f62 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.202842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f86b2c-a7a3-4463-b2c3-a8fb0de61f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.203942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f86b2c-a7a3-4463-b2c3-a8fb0de61f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.205790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f86b2c-a7a3-4463-b2c3-a8fb0de61f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.207717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f86b2c-a7a3-4463-b2c3-a8fb0de61f62 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.225820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 950/1873] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:25.236511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f86b2c-a7a3-4463-b2c3-a8fb0de61f62 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.253561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 924/1874] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:25.255686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f86b2c-a7a3-4463-b2c3-a8fb0de61f62 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.258098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022af046-a336-4159-b0b4-437085000dfc admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.278912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:25.362495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022af046-a336-4159-b0b4-437085000dfc admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.369643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b75eb-17d9-4d63-ac31-f3679adbd533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.370381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3015-7bf4-4d02-8420-3cc006511110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.370510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b75eb-17d9-4d63-ac31-f3679adbd533 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.370807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b75eb-17d9-4d63-ac31-f3679adbd533 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.371196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b75eb-17d9-4d63-ac31-f3679adbd533 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.371420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3015-7bf4-4d02-8420-3cc006511110 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.371759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3015-7bf4-4d02-8420-3cc006511110 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.372046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc3015-7bf4-4d02-8420-3cc006511110 admin admin] PATH_INFO: `/v3/projects/d5e95266554a46e191fa7f91f41af4da/users/d8f7e245f3a44960927e1ea23af309b9/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.372344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b75eb-17d9-4d63-ac31-f3679adbd533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.374358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b75eb-17d9-4d63-ac31-f3679adbd533 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.389222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3015-7bf4-4d02-8420-3cc006511110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e95266554a46e191fa7f91f41af4da, user_id=d8f7e245f3a44960927e1ea23af309b9, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.392168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc3015-7bf4-4d02-8420-3cc006511110 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.491175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 925/1875] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:25.492453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b75eb-17d9-4d63-ac31-f3679adbd533 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.502525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b75eb-17d9-4d63-ac31-f3679adbd533 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.507264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b31548-93aa-4c52-a991-1cc9e235b11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.507953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b31548-93aa-4c52-a991-1cc9e235b11b admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.508269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b31548-93aa-4c52-a991-1cc9e235b11b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.508683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b31548-93aa-4c52-a991-1cc9e235b11b admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.510670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b31548-93aa-4c52-a991-1cc9e235b11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.512370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b31548-93aa-4c52-a991-1cc9e235b11b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.517774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 951/1876] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:25 2022] PUT /identity/v3/projects/d5e95266554a46e191fa7f91f41af4da/users/d8f7e245f3a44960927e1ea23af309b9/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:25.527824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 926/1877] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:25.529779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3015-7bf4-4d02-8420-3cc006511110 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.530347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3b893-7540-4ca4-9578-f120a334c9cf None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.530638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3b893-7540-4ca4-9578-f120a334c9cf None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.530917 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3b893-7540-4ca4-9578-f120a334c9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.561892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-92f3b893-7540-4ca4-9578-f120a334c9cf None None] MFA Rules not processed for user `d8f7e245f3a44960927e1ea23af309b9`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:25.563168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-92f3b893-7540-4ca4-9578-f120a334c9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.600637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc3015-7bf4-4d02-8420-3cc006511110 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.606164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea42784-6f94-40c3-be77-b2651ed99f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.606943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea42784-6f94-40c3-be77-b2651ed99f00 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.607302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea42784-6f94-40c3-be77-b2651ed99f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.607652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea42784-6f94-40c3-be77-b2651ed99f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.608507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea42784-6f94-40c3-be77-b2651ed99f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.610400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea42784-6f94-40c3-be77-b2651ed99f00 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.622507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 952/1878] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:25.626836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea42784-6f94-40c3-be77-b2651ed99f00 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.637739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 927/1879] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:25.638289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea42784-6f94-40c3-be77-b2651ed99f00 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.641970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3b893-7540-4ca4-9578-f120a334c9cf None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.643531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0986176-6b6d-464f-b5a7-08ede9f1d307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.644472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0986176-6b6d-464f-b5a7-08ede9f1d307 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.644826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0986176-6b6d-464f-b5a7-08ede9f1d307 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.645111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0986176-6b6d-464f-b5a7-08ede9f1d307 admin admin] PATH_INFO: `/v3/projects/b9216170c1f541be9d43879680c6b145/users/bf94b8632ece4847966efead250a24c3/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.658174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3b893-7540-4ca4-9578-f120a334c9cf None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.662840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0986176-6b6d-464f-b5a7-08ede9f1d307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9216170c1f541be9d43879680c6b145, user_id=bf94b8632ece4847966efead250a24c3, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.666580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dd75d-7edd-44b6-8fcf-fbb7ad089fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.666580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dd75d-7edd-44b6-8fcf-fbb7ad089fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.666580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dd75d-7edd-44b6-8fcf-fbb7ad089fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.666580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76dd75d-7edd-44b6-8fcf-fbb7ad089fd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.666994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0986176-6b6d-464f-b5a7-08ede9f1d307 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.667082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dd75d-7edd-44b6-8fcf-fbb7ad089fd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.668669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76dd75d-7edd-44b6-8fcf-fbb7ad089fd3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.767448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 953/1880] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:25 2022] PUT /identity/v3/projects/b9216170c1f541be9d43879680c6b145/users/bf94b8632ece4847966efead250a24c3/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:25.779951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae1b46-13c1-486a-88da-24d4a05dd0d8 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.780274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae1b46-13c1-486a-88da-24d4a05dd0d8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.780559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae1b46-13c1-486a-88da-24d4a05dd0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.797286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 928/1881] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:25.802301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dd75d-7edd-44b6-8fcf-fbb7ad089fd3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.809005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-46ae1b46-13c1-486a-88da-24d4a05dd0d8 None None] MFA Rules not processed for user `bf94b8632ece4847966efead250a24c3`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:25.810377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-46ae1b46-13c1-486a-88da-24d4a05dd0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.870466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76dd75d-7edd-44b6-8fcf-fbb7ad089fd3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.875177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a4d2-ca1b-4ac0-8571-5c61d770006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.875853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a4d2-ca1b-4ac0-8571-5c61d770006a admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.876174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a4d2-ca1b-4ac0-8571-5c61d770006a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.876459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059a4d2-ca1b-4ac0-8571-5c61d770006a admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.877149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059a4d2-ca1b-4ac0-8571-5c61d770006a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.879080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059a4d2-ca1b-4ac0-8571-5c61d770006a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.879193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 954/1882] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:25.891981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae1b46-13c1-486a-88da-24d4a05dd0d8 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.893407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-46ae1b46-13c1-486a-88da-24d4a05dd0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:25.911374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:25.952076 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae1b46-13c1-486a-88da-24d4a05dd0d8 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.955873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa90f6c-55fc-4b9c-968b-c792afe2462b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.956556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa90f6c-55fc-4b9c-968b-c792afe2462b service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.956832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 929/1883] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:25 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:25.956945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa90f6c-55fc-4b9c-968b-c792afe2462b service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.961722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa90f6c-55fc-4b9c-968b-c792afe2462b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.961956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a4d2-ca1b-4ac0-8571-5c61d770006a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:25.968089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa90f6c-55fc-4b9c-968b-c792afe2462b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.969692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa90f6c-55fc-4b9c-968b-c792afe2462b service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.971710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059a4d2-ca1b-4ac0-8571-5c61d770006a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:25.975886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:25.978772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a6adf-7e05-458a-b54b-fed344aeeec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:25.979444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a6adf-7e05-458a-b54b-fed344aeeec2 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:25.979727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a6adf-7e05-458a-b54b-fed344aeeec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:25.980002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a6adf-7e05-458a-b54b-fed344aeeec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:25.980824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a6adf-7e05-458a-b54b-fed344aeeec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:25.980950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 955/1884] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:25.983021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a6adf-7e05-458a-b54b-fed344aeeec2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:25.993147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 930/1885] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:26.003827 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a6adf-7e05-458a-b54b-fed344aeeec2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.003932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa90f6c-55fc-4b9c-968b-c792afe2462b service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.011059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a6adf-7e05-458a-b54b-fed344aeeec2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.011835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa90f6c-55fc-4b9c-968b-c792afe2462b service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.015810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804256b8-aea8-482d-8fbc-a911ff3b0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.016633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.016769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804256b8-aea8-482d-8fbc-a911ff3b0a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.017055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804256b8-aea8-482d-8fbc-a911ff3b0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.017394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804256b8-aea8-482d-8fbc-a911ff3b0a01 admin admin] PATH_INFO: `/v3/projects/ba2bf19ebfa54efbb33ed1f4d49cfa12/users/bd31de4cd1624f75817cae2cecd2eabc/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.017492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.018099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.018330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] PATH_INFO: `/v3/users/bf94b8632ece4847966efead250a24c3` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.019505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf94b8632ece4847966efead250a24c3)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.021464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.035012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804256b8-aea8-482d-8fbc-a911ff3b0a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2bf19ebfa54efbb33ed1f4d49cfa12, user_id=bd31de4cd1624f75817cae2cecd2eabc, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.037735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804256b8-aea8-482d-8fbc-a911ff3b0a01 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.140980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 931/1886] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:25 2022] PUT /identity/v3/projects/ba2bf19ebfa54efbb33ed1f4d49cfa12/users/bd31de4cd1624f75817cae2cecd2eabc/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:26.145032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804256b8-aea8-482d-8fbc-a911ff3b0a01 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.213185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804256b8-aea8-482d-8fbc-a911ff3b0a01 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.218644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23361bf0-23fa-43e0-b061-56cb08b0463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.221723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23361bf0-23fa-43e0-b061-56cb08b0463d admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.221723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23361bf0-23fa-43e0-b061-56cb08b0463d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.221723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23361bf0-23fa-43e0-b061-56cb08b0463d admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.221723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23361bf0-23fa-43e0-b061-56cb08b0463d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.223265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23361bf0-23fa-43e0-b061-56cb08b0463d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.236026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 932/1887] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:26 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:26.238523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf94b8632ece4847966efead250a24c3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.240080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23361bf0-23fa-43e0-b061-56cb08b0463d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.251421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23361bf0-23fa-43e0-b061-56cb08b0463d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.256419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe3e2a-0b2b-4c56-bd43-579ad3374b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.257359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe3e2a-0b2b-4c56-bd43-579ad3374b45 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.257641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe3e2a-0b2b-4c56-bd43-579ad3374b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.257916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe3e2a-0b2b-4c56-bd43-579ad3374b45 admin admin] PATH_INFO: `/v3/projects/ba2bf19ebfa54efbb33ed1f4d49cfa12/users/bd31de4cd1624f75817cae2cecd2eabc/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.260770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe3e2a-0b2b-4c56-bd43-579ad3374b45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2bf19ebfa54efbb33ed1f4d49cfa12, user_id=bd31de4cd1624f75817cae2cecd2eabc, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.263410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe3e2a-0b2b-4c56-bd43-579ad3374b45 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.280846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 933/1888] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:26 2022] PUT /identity/v3/projects/ba2bf19ebfa54efbb33ed1f4d49cfa12/users/bd31de4cd1624f75817cae2cecd2eabc/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:26.285626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf5d9-9a9f-4199-8feb-e00353a8b326 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.285922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf5d9-9a9f-4199-8feb-e00353a8b326 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.286205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20bf5d9-9a9f-4199-8feb-e00353a8b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.313628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b20bf5d9-9a9f-4199-8feb-e00353a8b326 None None] MFA Rules not processed for user `bd31de4cd1624f75817cae2cecd2eabc`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:26.314915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b20bf5d9-9a9f-4199-8feb-e00353a8b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.322165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf94b8632ece4847966efead250a24c3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.322842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf94b8632ece4847966efead250a24c3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.330657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf94b8632ece4847966efead250a24c3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.331424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6552454b-a899-403f-9e5c-918a5bcfad4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf94b8632ece4847966efead250a24c3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:26.343458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:26.351855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 956/1889] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:25 2022] DELETE /identity/v3/users/bf94b8632ece4847966efead250a24c3 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:26.357078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6884d9a-1c5f-450a-908d-7b19f6905988 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.357463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6884d9a-1c5f-450a-908d-7b19f6905988 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.357817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6884d9a-1c5f-450a-908d-7b19f6905988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.380147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d6884d9a-1c5f-450a-908d-7b19f6905988 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:26.381851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d6884d9a-1c5f-450a-908d-7b19f6905988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.398515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:26.408150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 934/1890] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:26.416309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bf5d9-9a9f-4199-8feb-e00353a8b326 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.419658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b20bf5d9-9a9f-4199-8feb-e00353a8b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.450547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 957/1891] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:26.460465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6884d9a-1c5f-450a-908d-7b19f6905988 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.463136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d6884d9a-1c5f-450a-908d-7b19f6905988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.490171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20bf5d9-9a9f-4199-8feb-e00353a8b326 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.538111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6884d9a-1c5f-450a-908d-7b19f6905988 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.542574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.543230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.544694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.545517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.551131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.551954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.552780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.553063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.559485 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.560967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.574914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 958/1892] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:26.627961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.629511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.641309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 935/1893] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:26.754389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:26.755989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.775049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e3622-70aa-451c-a919-c3853f807ed3 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:26.780228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ceb7b8-347a-49c6-95a8-e051ce1030ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:26.780956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceb7b8-347a-49c6-95a8-e051ce1030ed service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:26.781249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceb7b8-347a-49c6-95a8-e051ce1030ed service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:26.781650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceb7b8-347a-49c6-95a8-e051ce1030ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:26.784153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-08ceb7b8-347a-49c6-95a8-e051ce1030ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:26.866807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ceb7b8-347a-49c6-95a8-e051ce1030ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:26.869249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ceb7b8-347a-49c6-95a8-e051ce1030ed service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:26.881753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 959/1894] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:27.208292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:27.210240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:27.230980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2877dd-5be8-4729-98d8-00e6a0bd0e81 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:27.239127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:27.239127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.239127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.239127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] PATH_INFO: `/v3/projects/b9216170c1f541be9d43879680c6b145` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.239801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9216170c1f541be9d43879680c6b145)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:27.241722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:27.373624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9216170c1f541be9d43879680c6b145', 'request_id': 'req-7107b600-0f4c-4c8a-8167-a104621b1b04'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:27.489883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9216170c1f541be9d43879680c6b145', 'request_id': 'req-7107b600-0f4c-4c8a-8167-a104621b1b04'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:27.498114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:27.504935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9216170c1f541be9d43879680c6b145', 'request_id': 'req-7107b600-0f4c-4c8a-8167-a104621b1b04'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:27.531150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] The token cache is being invalidate because project b9216170c1f541be9d43879680c6b145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:27.531150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:27.531150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7107b600-0f4c-4c8a-8167-a104621b1b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:27.539063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 936/1895] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:27 2022] DELETE /identity/v3/projects/b9216170c1f541be9d43879680c6b145 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:27.539465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c00f-9e31-42ec-9325-258c81b61f39 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.541678 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c00f-9e31-42ec-9325-258c81b61f39 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.541949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7c00f-9e31-42ec-9325-258c81b61f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.596350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-6fd7c00f-9e31-42ec-9325-258c81b61f39 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:27.599570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6fd7c00f-9e31-42ec-9325-258c81b61f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:27.678347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:27.734229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 960/1896] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:27.742096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53f9d-5845-4c9a-9539-dc93e582760c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.742568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53f9d-5845-4c9a-9539-dc93e582760c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.742644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53f9d-5845-4c9a-9539-dc93e582760c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.797418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7fc53f9d-5845-4c9a-9539-dc93e582760c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:27.800545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7fc53f9d-5845-4c9a-9539-dc93e582760c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:27.824896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 937/1897] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:27.826637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7c00f-9e31-42ec-9325-258c81b61f39 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:27.841367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7c00f-9e31-42ec-9325-258c81b61f39 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:27.849220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:27.850248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.851100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.851450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.859845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:27.859845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:27.859845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:27.867290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 961/1898] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:27.872338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53f9d-5845-4c9a-9539-dc93e582760c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:27.885164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53f9d-5845-4c9a-9539-dc93e582760c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:27.890132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc4afb-5656-4fb4-8473-ad613677106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:27.890852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc4afb-5656-4fb4-8473-ad613677106e admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:27.891140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc4afb-5656-4fb4-8473-ad613677106e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:27.891421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc4afb-5656-4fb4-8473-ad613677106e admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:27.892349 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc4afb-5656-4fb4-8473-ad613677106e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:27.894273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc4afb-5656-4fb4-8473-ad613677106e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:27.973118 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:27.979098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:27.992818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274665f5-dcf6-456c-9ea0-5d61edc98a46 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:28.002059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:28.003375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 938/1899] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:27 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:28.005656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.005952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.006282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] PATH_INFO: `/v3/users/5b1b05dfe33f41529e60b8602b38e14e` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.007970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc4afb-5656-4fb4-8473-ad613677106e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:28.008410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1b05dfe33f41529e60b8602b38e14e)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:28.011467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:28.099219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc4afb-5656-4fb4-8473-ad613677106e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:28.104886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7527f9-8861-421f-96c6-75854dc856a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:28.105924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7527f9-8861-421f-96c6-75854dc856a7 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.106727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7527f9-8861-421f-96c6-75854dc856a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.107027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7527f9-8861-421f-96c6-75854dc856a7 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.108738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7527f9-8861-421f-96c6-75854dc856a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:28.109929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7527f9-8861-421f-96c6-75854dc856a7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:28.225340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 939/1900] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:28 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:28.226223 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1b05dfe33f41529e60b8602b38e14e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:28.231122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7527f9-8861-421f-96c6-75854dc856a7 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:28.245392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7527f9-8861-421f-96c6-75854dc856a7 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:28.251905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5b458-6e30-4777-8d05-b6be72a996a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:28.254778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5b458-6e30-4777-8d05-b6be72a996a2 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.254778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5b458-6e30-4777-8d05-b6be72a996a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.254778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5b458-6e30-4777-8d05-b6be72a996a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.255132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5b458-6e30-4777-8d05-b6be72a996a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:28.256936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5b458-6e30-4777-8d05-b6be72a996a2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:28.269506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 940/1901] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:28.275126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5b458-6e30-4777-8d05-b6be72a996a2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:28.287070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5b458-6e30-4777-8d05-b6be72a996a2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:28.294019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1b05dfe33f41529e60b8602b38e14e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:28.294566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:28.294781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1b05dfe33f41529e60b8602b38e14e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:28.295298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.295603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.295871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.296866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:28.300886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:28.307792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1b05dfe33f41529e60b8602b38e14e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:28.308663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-85346c49-a665-4112-86ad-2076c4db2fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1b05dfe33f41529e60b8602b38e14e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:28.315029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 941/1902] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:28.316002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:28.317855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:28.332281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371545ef-f042-4964-bbe3-742ee56d1991 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:28.335592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 962/1903] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:27 2022] DELETE /identity/v3/users/5b1b05dfe33f41529e60b8602b38e14e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:28.338840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a249-54c5-4203-bb9a-f0d7a1db23cb None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.339151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a249-54c5-4203-bb9a-f0d7a1db23cb None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.339509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c2a249-54c5-4203-bb9a-f0d7a1db23cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.366439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b1c2a249-54c5-4203-bb9a-f0d7a1db23cb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:28.367846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b1c2a249-54c5-4203-bb9a-f0d7a1db23cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:28.412103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75457cf-40ed-47b4-8ed0-741cd80f856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:28.413874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75457cf-40ed-47b4-8ed0-741cd80f856f admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.414171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75457cf-40ed-47b4-8ed0-741cd80f856f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.415351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75457cf-40ed-47b4-8ed0-741cd80f856f admin admin] PATH_INFO: `/v3/projects/b21e65c1c53f48c39dcf39b8d05a71be/users/9078f21cce334aea96c3e8339329bce4/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.444566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75457cf-40ed-47b4-8ed0-741cd80f856f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21e65c1c53f48c39dcf39b8d05a71be, user_id=9078f21cce334aea96c3e8339329bce4, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:28.448150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 963/1904] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:28.450517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75457cf-40ed-47b4-8ed0-741cd80f856f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:28.459222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2a249-54c5-4203-bb9a-f0d7a1db23cb None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:28.461415 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b1c2a249-54c5-4203-bb9a-f0d7a1db23cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:28.511417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:28.527913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c2a249-54c5-4203-bb9a-f0d7a1db23cb None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:28.534312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 942/1905] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:28 2022] PUT /identity/v3/projects/b21e65c1c53f48c39dcf39b8d05a71be/users/9078f21cce334aea96c3e8339329bce4/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:28.551379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794f9e-79a9-4954-a830-df9c8377edb7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.551731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794f9e-79a9-4954-a830-df9c8377edb7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.551953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794f9e-79a9-4954-a830-df9c8377edb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.575599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:28.587548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a1794f9e-79a9-4954-a830-df9c8377edb7 None None] MFA Rules not processed for user `9078f21cce334aea96c3e8339329bce4`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:28.587739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a1794f9e-79a9-4954-a830-df9c8377edb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:28.589349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:28.590098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.590442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.590782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.660867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:28.662501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:28.666339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 943/1906] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:28.675655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 964/1907] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:28.677393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1794f9e-79a9-4954-a830-df9c8377edb7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:28.678932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a1794f9e-79a9-4954-a830-df9c8377edb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:28.810189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1794f9e-79a9-4954-a830-df9c8377edb7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:28.822997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910594c-5583-404e-8be8-0fb2b2d2d966 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c0883bc26a04de2916c78df342d2023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:28.842227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910594c-5583-404e-8be8-0fb2b2d2d966 service glance] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:28.842721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910594c-5583-404e-8be8-0fb2b2d2d966 service glance] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:28.843781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910594c-5583-404e-8be8-0fb2b2d2d966 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:28.889486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910594c-5583-404e-8be8-0fb2b2d2d966 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:28.893315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2910594c-5583-404e-8be8-0fb2b2d2d966 service glance] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:28.923403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 944/1908] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:29.911318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:29.915888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:29.950993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c93550-62ed-4b16-89e8-3805b8b6e6d8 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:29.982392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:29.983571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:29.984155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:29.984635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] PATH_INFO: `/v3/projects/f7b940a05c91473eb2f7cd30484de91f` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:29.987245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7b940a05c91473eb2f7cd30484de91f)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:29.992066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:30.174243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7b940a05c91473eb2f7cd30484de91f', 'request_id': 'req-097ffe1c-843a-4d23-9185-92ac3c32c3f1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.270873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7b940a05c91473eb2f7cd30484de91f', 'request_id': 'req-097ffe1c-843a-4d23-9185-92ac3c32c3f1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.282451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7b940a05c91473eb2f7cd30484de91f', 'request_id': 'req-097ffe1c-843a-4d23-9185-92ac3c32c3f1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.302476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] The token cache is being invalidate because project f7b940a05c91473eb2f7cd30484de91f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:30.303188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.304138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.311951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 965/1909] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:29 2022] DELETE /identity/v3/projects/f7b940a05c91473eb2f7cd30484de91f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:30.312527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910594c-5583-404e-8be8-0fb2b2d2d966 service glance] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:30.327828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910594c-5583-404e-8be8-0fb2b2d2d966 service glance] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:30.434720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:30.435814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:30.435936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:30.436232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] PATH_INFO: `/v3/users/e41c7362b03c4a9ca3c5e117a83adec4` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:30.437497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41c7362b03c4a9ca3c5e117a83adec4)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:30.439679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:30.471152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:30.655818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41c7362b03c4a9ca3c5e117a83adec4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.771880 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41c7362b03c4a9ca3c5e117a83adec4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.773785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41c7362b03c4a9ca3c5e117a83adec4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.788249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41c7362b03c4a9ca3c5e117a83adec4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.789933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-36670074-fac9-46c1-ab05-5f8faa20540a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41c7362b03c4a9ca3c5e117a83adec4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:30.830458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 945/1910] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:30 2022] DELETE /identity/v3/users/e41c7362b03c4a9ca3c5e117a83adec4 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:31.115881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:31.117520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:31.146182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ffe1c-843a-4d23-9185-92ac3c32c3f1 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:31.214562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:31.261752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:31.263427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:31.263769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:31.264144 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] PATH_INFO: `/v3/users/d8f7e245f3a44960927e1ea23af309b9` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:31.265429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8f7e245f3a44960927e1ea23af309b9)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:31.267495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:31.520045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8f7e245f3a44960927e1ea23af309b9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.641050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8f7e245f3a44960927e1ea23af309b9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.641863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8f7e245f3a44960927e1ea23af309b9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.656692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8f7e245f3a44960927e1ea23af309b9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.660516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8f7e245f3a44960927e1ea23af309b9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:31.704517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 966/1911] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:31 2022] DELETE /identity/v3/users/d8f7e245f3a44960927e1ea23af309b9 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:31.709372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa6b6-8c17-479f-ba52-d2c1b1b33443 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:31.719060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa6b6-8c17-479f-ba52-d2c1b1b33443 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:31.719060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fa6b6-8c17-479f-ba52-d2c1b1b33443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:31.732330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-688fa6b6-8c17-479f-ba52-d2c1b1b33443 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:31.733560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-688fa6b6-8c17-479f-ba52-d2c1b1b33443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:31.806720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:31.807659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:31.850528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 946/1912] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:31.868770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fa6b6-8c17-479f-ba52-d2c1b1b33443 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:31.870438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-688fa6b6-8c17-479f-ba52-d2c1b1b33443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:31.957964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a871e5-7a4d-4187-84d7-a4fa2127240a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:31.971336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:31.972286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:31.972579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:31.972871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] PATH_INFO: `/v3/projects/86b38e015e404f71901d84f1bd38e0bd` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:31.974528 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86b38e015e404f71901d84f1bd38e0bd)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:31.978292 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:31.997674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688fa6b6-8c17-479f-ba52-d2c1b1b33443 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.007402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02310b0-a492-42fe-bcb6-49b8c494c4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.009034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02310b0-a492-42fe-bcb6-49b8c494c4bd service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.009769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02310b0-a492-42fe-bcb6-49b8c494c4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.010237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02310b0-a492-42fe-bcb6-49b8c494c4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.025525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02310b0-a492-42fe-bcb6-49b8c494c4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.027152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02310b0-a492-42fe-bcb6-49b8c494c4bd service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.038833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 947/1913] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:32.156641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:32.159318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86b38e015e404f71901d84f1bd38e0bd', 'request_id': 'req-de08d016-c5ba-4727-8d50-876c592e1927'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.281917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86b38e015e404f71901d84f1bd38e0bd', 'request_id': 'req-de08d016-c5ba-4727-8d50-876c592e1927'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.302481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86b38e015e404f71901d84f1bd38e0bd', 'request_id': 'req-de08d016-c5ba-4727-8d50-876c592e1927'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.327436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] The token cache is being invalidate because project 86b38e015e404f71901d84f1bd38e0bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:32.328170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.329187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de08d016-c5ba-4727-8d50-876c592e1927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.347800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 967/1914] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:31 2022] DELETE /identity/v3/projects/86b38e015e404f71901d84f1bd38e0bd => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:32.354707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0511dd4-71f6-45ea-8644-af65fa0819f8 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.355214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0511dd4-71f6-45ea-8644-af65fa0819f8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.355650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0511dd4-71f6-45ea-8644-af65fa0819f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.388003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d0511dd4-71f6-45ea-8644-af65fa0819f8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:32.390184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d0511dd4-71f6-45ea-8644-af65fa0819f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.447569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:32.492326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 948/1915] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:32.501653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1f1a5-881a-44dd-9abb-6d63f3c67a2b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.501942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1f1a5-881a-44dd-9abb-6d63f3c67a2b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.502228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1f1a5-881a-44dd-9abb-6d63f3c67a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.523010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-33b1f1a5-881a-44dd-9abb-6d63f3c67a2b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:32.525051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-33b1f1a5-881a-44dd-9abb-6d63f3c67a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.534009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0511dd4-71f6-45ea-8644-af65fa0819f8 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.538412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d0511dd4-71f6-45ea-8644-af65fa0819f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.543518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 968/1916] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:32.544582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1f1a5-881a-44dd-9abb-6d63f3c67a2b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.556741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1f1a5-881a-44dd-9abb-6d63f3c67a2b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.558825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0511dd4-71f6-45ea-8644-af65fa0819f8 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.565949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.565949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.566738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.566738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] PATH_INFO: `/v3/projects/d5e95266554a46e191fa7f91f41af4da` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.569186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e95266554a46e191fa7f91f41af4da)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.571652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.576243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.577049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.577411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.577835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.578625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:32.579012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.581154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.599486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 969/1917] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:32.607715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.624522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08c3de-9d78-46d8-8d18-a2f5588f41c2 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.635407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b083f-dc39-476e-94c8-7647a01d6f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.636249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b083f-dc39-476e-94c8-7647a01d6f3f admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.636610 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b083f-dc39-476e-94c8-7647a01d6f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.637127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b083f-dc39-476e-94c8-7647a01d6f3f admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.637938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b083f-dc39-476e-94c8-7647a01d6f3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.640324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b083f-dc39-476e-94c8-7647a01d6f3f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.720029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e95266554a46e191fa7f91f41af4da', 'request_id': 'req-3fd5973b-4949-4d97-8a58-f1620c62a77f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.732997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 970/1918] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:32 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:32.737036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b083f-dc39-476e-94c8-7647a01d6f3f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.817012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b083f-dc39-476e-94c8-7647a01d6f3f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.819347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e95266554a46e191fa7f91f41af4da', 'request_id': 'req-3fd5973b-4949-4d97-8a58-f1620c62a77f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.823921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.824984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.825348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.825634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.826355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.828356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.830042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e95266554a46e191fa7f91f41af4da', 'request_id': 'req-3fd5973b-4949-4d97-8a58-f1620c62a77f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.847541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] The token cache is being invalidate because project d5e95266554a46e191fa7f91f41af4da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:32.848778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.850165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:32.854485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 949/1919] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:32 2022] DELETE /identity/v3/projects/d5e95266554a46e191fa7f91f41af4da => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:32.859303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.861770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.930151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 971/1920] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:32 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:32.934392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.936438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:32.948455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5973b-4949-4d97-8a58-f1620c62a77f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.951302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493ec79-55e8-4b5b-b04e-06b9032b2cab admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:32.957253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378773a-a373-45c1-8b39-521515b14081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.957681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:32.959706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.959996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.960305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] PATH_INFO: `/v3/users/bd31de4cd1624f75817cae2cecd2eabc` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.960467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378773a-a373-45c1-8b39-521515b14081 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:32.960764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378773a-a373-45c1-8b39-521515b14081 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:32.961136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378773a-a373-45c1-8b39-521515b14081 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:32.962860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378773a-a373-45c1-8b39-521515b14081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.964633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378773a-a373-45c1-8b39-521515b14081 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.966541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd31de4cd1624f75817cae2cecd2eabc)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:32.969723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:32.979007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 972/1921] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:32.984009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378773a-a373-45c1-8b39-521515b14081 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:32.997663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378773a-a373-45c1-8b39-521515b14081 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.003555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a062f90-e407-427e-b635-96ed308ff6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.004458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062f90-e407-427e-b635-96ed308ff6fa admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.004897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062f90-e407-427e-b635-96ed308ff6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.005186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a062f90-e407-427e-b635-96ed308ff6fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.006572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a062f90-e407-427e-b635-96ed308ff6fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.008934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a062f90-e407-427e-b635-96ed308ff6fa admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.021989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 973/1922] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:33.027004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a062f90-e407-427e-b635-96ed308ff6fa admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.037635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a062f90-e407-427e-b635-96ed308ff6fa admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.042561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a9c06-d844-4951-9342-984b2fa05b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.043473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a9c06-d844-4951-9342-984b2fa05b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.043796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a9c06-d844-4951-9342-984b2fa05b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.044081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a9c06-d844-4951-9342-984b2fa05b9f admin admin] PATH_INFO: `/v3/projects/3f1cb0825d304d4eb5613e80a21b34af/users/1b5b73133313460eb6c76b403ef32d07/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.061041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a9c06-d844-4951-9342-984b2fa05b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1cb0825d304d4eb5613e80a21b34af, user_id=1b5b73133313460eb6c76b403ef32d07, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.063971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a9c06-d844-4951-9342-984b2fa05b9f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.151842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 974/1923] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:33 2022] PUT /identity/v3/projects/3f1cb0825d304d4eb5613e80a21b34af/users/1b5b73133313460eb6c76b403ef32d07/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:33.163141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8158-5116-4db6-bf23-52271285365b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.163703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8158-5116-4db6-bf23-52271285365b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.163703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b8158-5116-4db6-bf23-52271285365b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.217063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2a7b8158-5116-4db6-bf23-52271285365b None None] MFA Rules not processed for user `1b5b73133313460eb6c76b403ef32d07`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:33.219345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2a7b8158-5116-4db6-bf23-52271285365b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:33.247909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd31de4cd1624f75817cae2cecd2eabc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:33.255428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:33.316945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 975/1924] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:33.321189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8158-5116-4db6-bf23-52271285365b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.344614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd31de4cd1624f75817cae2cecd2eabc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:33.349269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd31de4cd1624f75817cae2cecd2eabc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:33.356515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd31de4cd1624f75817cae2cecd2eabc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:33.358484 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd31de4cd1624f75817cae2cecd2eabc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:33.375455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:33.389795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 950/1925] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:32 2022] DELETE /identity/v3/users/bd31de4cd1624f75817cae2cecd2eabc => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:33.415344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b8158-5116-4db6-bf23-52271285365b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.526373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.527070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.527372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.527691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.529085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.533078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.687677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 976/1926] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:33 2022] POST /identity/v3/projects => generated 387 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:33.688524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.689813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:33.714390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f99234-2cd2-402e-b793-929c62479941 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.837177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa3aff-7ead-4f63-8409-dfdfc0e4a9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.837869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa3aff-7ead-4f63-8409-dfdfc0e4a9fd admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.838165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa3aff-7ead-4f63-8409-dfdfc0e4a9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.838449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa3aff-7ead-4f63-8409-dfdfc0e4a9fd admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.839163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa3aff-7ead-4f63-8409-dfdfc0e4a9fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.843363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aa3aff-7ead-4f63-8409-dfdfc0e4a9fd admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:33.959105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.959904 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 951/1927] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:33 2022] POST /identity/v3/users => generated 300 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:33.965147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:33.966096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa3aff-7ead-4f63-8409-dfdfc0e4a9fd admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:33.985659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa3aff-7ead-4f63-8409-dfdfc0e4a9fd admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:33.990967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0434ecb-940f-4424-ab6f-944b60e852cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:33.991758 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434ecb-940f-4424-ab6f-944b60e852cb admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:33.992437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434ecb-940f-4424-ab6f-944b60e852cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:33.993629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0434ecb-940f-4424-ab6f-944b60e852cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:33.995035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0434ecb-940f-4424-ab6f-944b60e852cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:33.997513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0434ecb-940f-4424-ab6f-944b60e852cb admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.012059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622cad1-126d-4119-b37c-ae338ef0bd41 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.015224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 952/1928] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:34.020087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0434ecb-940f-4424-ab6f-944b60e852cb admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.022094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.022982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.023267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.023546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] PATH_INFO: `/v3/projects/ba2bf19ebfa54efbb33ed1f4d49cfa12` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.025761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2bf19ebfa54efbb33ed1f4d49cfa12)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.027937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.037107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0434ecb-940f-4424-ab6f-944b60e852cb admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.044068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d306d1-6153-4ebd-b34b-552a951584ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.045087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d306d1-6153-4ebd-b34b-552a951584ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.045445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d306d1-6153-4ebd-b34b-552a951584ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.045798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d306d1-6153-4ebd-b34b-552a951584ec admin admin] PATH_INFO: `/v3/projects/c6d2f196c1a8417ba80a0ade227d548d/users/25d0e8f596fc43338752a16011969d03/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.064213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d306d1-6153-4ebd-b34b-552a951584ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d2f196c1a8417ba80a0ade227d548d, user_id=25d0e8f596fc43338752a16011969d03, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.067095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d306d1-6153-4ebd-b34b-552a951584ec admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.187159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 953/1929] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:34 2022] PUT /identity/v3/projects/c6d2f196c1a8417ba80a0ade227d548d/users/25d0e8f596fc43338752a16011969d03/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:34.191185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d306d1-6153-4ebd-b34b-552a951584ec admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.200957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2bf19ebfa54efbb33ed1f4d49cfa12', 'request_id': 'req-eeef3a62-213a-4605-a030-a5b0721f6960'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.205575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:34.265805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d306d1-6153-4ebd-b34b-552a951584ec admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.312877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:34.326928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e4fb6-ec1e-4b63-a6bf-a552c09129a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.327765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4fb6-ec1e-4b63-a6bf-a552c09129a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.328173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4fb6-ec1e-4b63-a6bf-a552c09129a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.331079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4fb6-ec1e-4b63-a6bf-a552c09129a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.331079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e4fb6-ec1e-4b63-a6bf-a552c09129a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.331079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e4fb6-ec1e-4b63-a6bf-a552c09129a4 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.347894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 954/1930] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:34.351177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e4fb6-ec1e-4b63-a6bf-a552c09129a4 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.351331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2bf19ebfa54efbb33ed1f4d49cfa12', 'request_id': 'req-eeef3a62-213a-4605-a030-a5b0721f6960'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.358454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2bf19ebfa54efbb33ed1f4d49cfa12', 'request_id': 'req-eeef3a62-213a-4605-a030-a5b0721f6960'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.363884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e4fb6-ec1e-4b63-a6bf-a552c09129a4 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.371361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eec4c-0706-41de-ba27-83be3fb944c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.372448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eec4c-0706-41de-ba27-83be3fb944c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.372868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eec4c-0706-41de-ba27-83be3fb944c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.373163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eec4c-0706-41de-ba27-83be3fb944c3 admin admin] PATH_INFO: `/v3/projects/c6d2f196c1a8417ba80a0ade227d548d/users/25d0e8f596fc43338752a16011969d03/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.373283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] The token cache is being invalidate because project ba2bf19ebfa54efbb33ed1f4d49cfa12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:34.373536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.374444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-eeef3a62-213a-4605-a030-a5b0721f6960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:34.385991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 977/1931] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:33 2022] DELETE /identity/v3/projects/ba2bf19ebfa54efbb33ed1f4d49cfa12 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:34.386560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eec4c-0706-41de-ba27-83be3fb944c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d2f196c1a8417ba80a0ade227d548d, user_id=25d0e8f596fc43338752a16011969d03, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.386691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563e753-7191-4bd7-8ddf-e10313c2b842 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.386691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563e753-7191-4bd7-8ddf-e10313c2b842 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.386691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563e753-7191-4bd7-8ddf-e10313c2b842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.387260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eec4c-0706-41de-ba27-83be3fb944c3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.409335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 955/1932] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:34 2022] PUT /identity/v3/projects/c6d2f196c1a8417ba80a0ade227d548d/users/25d0e8f596fc43338752a16011969d03/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:34.416363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1563e753-7191-4bd7-8ddf-e10313c2b842 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:34.418248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1563e753-7191-4bd7-8ddf-e10313c2b842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.420315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45448d5c-21a3-4ea0-9a99-6d09cdb0e4ea None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.421204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45448d5c-21a3-4ea0-9a99-6d09cdb0e4ea None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.421653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45448d5c-21a3-4ea0-9a99-6d09cdb0e4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.461154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-45448d5c-21a3-4ea0-9a99-6d09cdb0e4ea None None] MFA Rules not processed for user `25d0e8f596fc43338752a16011969d03`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:34.463302 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-45448d5c-21a3-4ea0-9a99-6d09cdb0e4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.500399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 978/1933] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:34.510776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1b987-03eb-46cc-90a9-c7f045196f9b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.511779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1b987-03eb-46cc-90a9-c7f045196f9b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.513034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1b987-03eb-46cc-90a9-c7f045196f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.544344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-8be1b987-03eb-46cc-90a9-c7f045196f9b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:34.545580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8be1b987-03eb-46cc-90a9-c7f045196f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.564107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 979/1934] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:34.567775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1b987-03eb-46cc-90a9-c7f045196f9b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.576534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 956/1935] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:34.583969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1b987-03eb-46cc-90a9-c7f045196f9b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.588483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45448d5c-21a3-4ea0-9a99-6d09cdb0e4ea None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.589965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-45448d5c-21a3-4ea0-9a99-6d09cdb0e4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.590739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.591513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.591874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.592260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.593311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:34.593700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.596504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.613811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 980/1936] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:34.618813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.631261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484d889-fe72-45be-87a0-a2149eabb79f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.638729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b2073-9b2a-43dc-9ddd-3a0cf592e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.639503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b2073-9b2a-43dc-9ddd-3a0cf592e916 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.639789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b2073-9b2a-43dc-9ddd-3a0cf592e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.640068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5b2073-9b2a-43dc-9ddd-3a0cf592e916 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.642549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b2073-9b2a-43dc-9ddd-3a0cf592e916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.657488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5b2073-9b2a-43dc-9ddd-3a0cf592e916 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.685276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45448d5c-21a3-4ea0-9a99-6d09cdb0e4ea None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.689170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.690241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.690662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.691062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.702506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.703959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.713801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 957/1937] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:34.740835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.741601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 981/1938] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:34.745928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:34.746079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b2073-9b2a-43dc-9ddd-3a0cf592e916 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.761022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31df065-ae4f-4c7b-a7a4-979234fd580f service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.811959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5b2073-9b2a-43dc-9ddd-3a0cf592e916 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.816733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e74a4-8b46-41b4-a2e2-f405c6460430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.817413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e74a4-8b46-41b4-a2e2-f405c6460430 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.817703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e74a4-8b46-41b4-a2e2-f405c6460430 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.817986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13e74a4-8b46-41b4-a2e2-f405c6460430 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.818684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e74a4-8b46-41b4-a2e2-f405c6460430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.821445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13e74a4-8b46-41b4-a2e2-f405c6460430 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.825017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.825898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.826264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.826569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] PATH_INFO: `/v3/users/1b5b73133313460eb6c76b403ef32d07` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.827975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b5b73133313460eb6c76b403ef32d07)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.830030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.833702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:34.900098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 982/1939] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:34 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:34.904041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e74a4-8b46-41b4-a2e2-f405c6460430 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.915283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13e74a4-8b46-41b4-a2e2-f405c6460430 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.920402 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e95c4-6b3a-468a-b935-b63bdc7f90b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.921091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e95c4-6b3a-468a-b935-b63bdc7f90b6 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.921490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e95c4-6b3a-468a-b935-b63bdc7f90b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.921773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e95c4-6b3a-468a-b935-b63bdc7f90b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.922519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e95c4-6b3a-468a-b935-b63bdc7f90b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.924201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14e95c4-6b3a-468a-b935-b63bdc7f90b6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.935478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 983/1940] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:34.940026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e95c4-6b3a-468a-b935-b63bdc7f90b6 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.950380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e95c4-6b3a-468a-b935-b63bdc7f90b6 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.955328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dfd53-18ba-42e0-8c88-0a010c002948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.956049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dfd53-18ba-42e0-8c88-0a010c002948 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.956368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dfd53-18ba-42e0-8c88-0a010c002948 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.956648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808dfd53-18ba-42e0-8c88-0a010c002948 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.957374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808dfd53-18ba-42e0-8c88-0a010c002948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:34.958987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808dfd53-18ba-42e0-8c88-0a010c002948 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:34.970774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 984/1941] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:34.974847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dfd53-18ba-42e0-8c88-0a010c002948 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:34.985392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808dfd53-18ba-42e0-8c88-0a010c002948 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:34.989930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759bc991-bc65-4d4f-9a33-674f4335091e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:34.990790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bc991-bc65-4d4f-9a33-674f4335091e admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:34.991068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bc991-bc65-4d4f-9a33-674f4335091e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:34.991341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759bc991-bc65-4d4f-9a33-674f4335091e admin admin] PATH_INFO: `/v3/projects/3077c72cf2874f58bf91311d581450e9/users/c0552145127647279abaff5f1f347fe9/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:34.995023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:35.007755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bc991-bc65-4d4f-9a33-674f4335091e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3077c72cf2874f58bf91311d581450e9, user_id=c0552145127647279abaff5f1f347fe9, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:35.010656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759bc991-bc65-4d4f-9a33-674f4335091e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:35.014779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b5b73133313460eb6c76b403ef32d07'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:35.087037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 985/1942] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:34 2022] PUT /identity/v3/projects/3077c72cf2874f58bf91311d581450e9/users/c0552145127647279abaff5f1f347fe9/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:35.097334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31703a-2f2c-437e-aa68-0ad7e4ed5692 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:35.097656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31703a-2f2c-437e-aa68-0ad7e4ed5692 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:35.097936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31703a-2f2c-437e-aa68-0ad7e4ed5692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:35.110508 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b5b73133313460eb6c76b403ef32d07'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:35.111118 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b5b73133313460eb6c76b403ef32d07'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:35.120489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b5b73133313460eb6c76b403ef32d07'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:35.121161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-6aab1de0-8392-4159-a397-e9ec8c49e561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b5b73133313460eb6c76b403ef32d07'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:35.126755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fa31703a-2f2c-437e-aa68-0ad7e4ed5692 None None] MFA Rules not processed for user `c0552145127647279abaff5f1f347fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:35.128005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fa31703a-2f2c-437e-aa68-0ad7e4ed5692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:35.142413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 958/1943] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:34 2022] DELETE /identity/v3/users/1b5b73133313460eb6c76b403ef32d07 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:35.146816 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd920235-bac4-445a-90da-f040d137dc7a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:35.147107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd920235-bac4-445a-90da-f040d137dc7a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:35.147387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd920235-bac4-445a-90da-f040d137dc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:35.167105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-fd920235-bac4-445a-90da-f040d137dc7a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:35.168368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fd920235-bac4-445a-90da-f040d137dc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:35.195489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 986/1944] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:35.206650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31703a-2f2c-437e-aa68-0ad7e4ed5692 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:35.208345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fa31703a-2f2c-437e-aa68-0ad7e4ed5692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:35.231224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 959/1945] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:35.241201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd920235-bac4-445a-90da-f040d137dc7a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:35.242532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fd920235-bac4-445a-90da-f040d137dc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:35.271969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31703a-2f2c-437e-aa68-0ad7e4ed5692 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:35.303987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd920235-bac4-445a-90da-f040d137dc7a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:35.307868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:35.308554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:35.308838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:35.309237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:35.320572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:35.322185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:35.323376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5174468-f998-4d66-a56b-f8acfc4c9531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:35.324040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5174468-f998-4d66-a56b-f8acfc4c9531 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:35.324356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5174468-f998-4d66-a56b-f8acfc4c9531 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:35.324638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5174468-f998-4d66-a56b-f8acfc4c9531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:35.332303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 960/1946] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:35.370893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:35.383370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5174468-f998-4d66-a56b-f8acfc4c9531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:35.385158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5174468-f998-4d66-a56b-f8acfc4c9531 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:35.394321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 987/1947] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:35.760572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:35.772203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:35.787210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03485fb-3003-453e-94e5-b8b0103740e8 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:35.792452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:35.793311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:35.793573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:35.793859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] PATH_INFO: `/v3/projects/3f1cb0825d304d4eb5613e80a21b34af` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:35.795712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1cb0825d304d4eb5613e80a21b34af)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:35.797589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:35.816973 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:35.917109 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1cb0825d304d4eb5613e80a21b34af', 'request_id': 'req-d71699c5-969c-419e-8261-3718092c0441'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:35.989017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1cb0825d304d4eb5613e80a21b34af', 'request_id': 'req-d71699c5-969c-419e-8261-3718092c0441'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:35.996023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1cb0825d304d4eb5613e80a21b34af', 'request_id': 'req-d71699c5-969c-419e-8261-3718092c0441'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.007669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] The token cache is being invalidate because project 3f1cb0825d304d4eb5613e80a21b34af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:36.007954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.008653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.010913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 961/1948] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:35 2022] DELETE /identity/v3/projects/3f1cb0825d304d4eb5613e80a21b34af => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:36.014954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5174468-f998-4d66-a56b-f8acfc4c9531 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:36.027469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5174468-f998-4d66-a56b-f8acfc4c9531 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:36.086878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:36.088899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:36.089115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:36.089674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] PATH_INFO: `/v3/users/25d0e8f596fc43338752a16011969d03` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:36.094838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d0e8f596fc43338752a16011969d03)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:36.097084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:36.304201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d0e8f596fc43338752a16011969d03'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.367559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d0e8f596fc43338752a16011969d03'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.368221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d0e8f596fc43338752a16011969d03'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.375929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d0e8f596fc43338752a16011969d03'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.376615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-892f8273-2098-44fb-9512-d6baa21fe106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d0e8f596fc43338752a16011969d03'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:36.394418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 988/1949] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:36 2022] DELETE /identity/v3/users/25d0e8f596fc43338752a16011969d03 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:36.871711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:36.873253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:36.895928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71699c5-969c-419e-8261-3718092c0441 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:36.972527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:36.974283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:36.974283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:36.974606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] PATH_INFO: `/v3/projects/c6d2f196c1a8417ba80a0ade227d548d` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:36.975940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d2f196c1a8417ba80a0ade227d548d)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:36.979195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.122829 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d2f196c1a8417ba80a0ade227d548d', 'request_id': 'req-00b39a54-3c08-457b-81f4-5315c1ebb1de'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.190260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d2f196c1a8417ba80a0ade227d548d', 'request_id': 'req-00b39a54-3c08-457b-81f4-5315c1ebb1de'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.197767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d2f196c1a8417ba80a0ade227d548d', 'request_id': 'req-00b39a54-3c08-457b-81f4-5315c1ebb1de'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.209712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] The token cache is being invalidate because project c6d2f196c1a8417ba80a0ade227d548d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:37.210030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.210723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-00b39a54-3c08-457b-81f4-5315c1ebb1de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:37.212581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 962/1950] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:36 2022] DELETE /identity/v3/projects/c6d2f196c1a8417ba80a0ade227d548d => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:37.220351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090b420-1a15-4d02-a872-59c191fb84cf None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.220351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090b420-1a15-4d02-a872-59c191fb84cf None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.220351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090b420-1a15-4d02-a872-59c191fb84cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.238817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1090b420-1a15-4d02-a872-59c191fb84cf None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:37.240183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1090b420-1a15-4d02-a872-59c191fb84cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:37.302487 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 989/1951] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:37.312728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fda96-1917-4cbd-bd2a-36bfcc6d28fa None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.313204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fda96-1917-4cbd-bd2a-36bfcc6d28fa None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.313583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fda96-1917-4cbd-bd2a-36bfcc6d28fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.334532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1f4fda96-1917-4cbd-bd2a-36bfcc6d28fa None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:37.335781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1f4fda96-1917-4cbd-bd2a-36bfcc6d28fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:37.344948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 963/1952] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:37.349347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090b420-1a15-4d02-a872-59c191fb84cf None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.355892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:37.359622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090b420-1a15-4d02-a872-59c191fb84cf None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.364534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.365207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.365497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.365780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.366520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:37.366835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.368552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.380616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 990/1953] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:37.385586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fda96-1917-4cbd-bd2a-36bfcc6d28fa None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.395133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fda96-1917-4cbd-bd2a-36bfcc6d28fa None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.401075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80078361-14b1-488c-b769-f22e34194f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.401831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80078361-14b1-488c-b769-f22e34194f7d admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.402260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80078361-14b1-488c-b769-f22e34194f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.402545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80078361-14b1-488c-b769-f22e34194f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.403240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80078361-14b1-488c-b769-f22e34194f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.405298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80078361-14b1-488c-b769-f22e34194f7d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.536534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 964/1954] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:37.541695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.553308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee65c3c-7f89-4fb4-9c40-94495121110e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.617003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2dacea-b5b7-41ef-8872-1a919f0c2660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.617548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dacea-b5b7-41ef-8872-1a919f0c2660 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.617835 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dacea-b5b7-41ef-8872-1a919f0c2660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.618113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dacea-b5b7-41ef-8872-1a919f0c2660 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.618901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2dacea-b5b7-41ef-8872-1a919f0c2660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.621577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2dacea-b5b7-41ef-8872-1a919f0c2660 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.738409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 991/1955] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:37.742797 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80078361-14b1-488c-b769-f22e34194f7d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.754402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80078361-14b1-488c-b769-f22e34194f7d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.759047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87503da-90c3-4f1b-8873-56f7700ea222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.760423 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87503da-90c3-4f1b-8873-56f7700ea222 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.760616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87503da-90c3-4f1b-8873-56f7700ea222 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.760860 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87503da-90c3-4f1b-8873-56f7700ea222 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.761592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87503da-90c3-4f1b-8873-56f7700ea222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.763206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87503da-90c3-4f1b-8873-56f7700ea222 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.764793 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:37.775232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 965/1956] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:37.780124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2dacea-b5b7-41ef-8872-1a919f0c2660 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.791678 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2dacea-b5b7-41ef-8872-1a919f0c2660 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.796948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34f62f-92b8-4a52-8137-58e46dd3e120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.797666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34f62f-92b8-4a52-8137-58e46dd3e120 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.801409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34f62f-92b8-4a52-8137-58e46dd3e120 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.801409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34f62f-92b8-4a52-8137-58e46dd3e120 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.801409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34f62f-92b8-4a52-8137-58e46dd3e120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.801409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34f62f-92b8-4a52-8137-58e46dd3e120 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.812544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 992/1957] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:37.816912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87503da-90c3-4f1b-8873-56f7700ea222 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:37.827197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87503da-90c3-4f1b-8873-56f7700ea222 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:37.832545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20c6b-5781-492f-8979-f724b75bfb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:37.833461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20c6b-5781-492f-8979-f724b75bfb04 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.833743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20c6b-5781-492f-8979-f724b75bfb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.834025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20c6b-5781-492f-8979-f724b75bfb04 admin admin] PATH_INFO: `/v3/projects/5429691aa9c04c97b3c68d9a4df13659/users/4394dbebd4e34164add908888891a143/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.851928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d20c6b-5781-492f-8979-f724b75bfb04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5429691aa9c04c97b3c68d9a4df13659, user_id=4394dbebd4e34164add908888891a143, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:37.855499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d20c6b-5781-492f-8979-f724b75bfb04 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:37.923547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 966/1958] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:37 2022] PUT /identity/v3/projects/5429691aa9c04c97b3c68d9a4df13659/users/4394dbebd4e34164add908888891a143/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:37.934624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebbd25-4e12-41de-8674-62d19f16b2ae None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:37.934919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebbd25-4e12-41de-8674-62d19f16b2ae None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:37.935200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ebbd25-4e12-41de-8674-62d19f16b2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:37.961843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-43ebbd25-4e12-41de-8674-62d19f16b2ae None None] MFA Rules not processed for user `4394dbebd4e34164add908888891a143`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:37.963066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-43ebbd25-4e12-41de-8674-62d19f16b2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:38.032243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 993/1959] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:38.036099 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20c6b-5781-492f-8979-f724b75bfb04 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.111521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20c6b-5781-492f-8979-f724b75bfb04 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.116293 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea966e2-9bef-49ef-9984-e60db39b9089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.116963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea966e2-9bef-49ef-9984-e60db39b9089 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.117247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea966e2-9bef-49ef-9984-e60db39b9089 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.117525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea966e2-9bef-49ef-9984-e60db39b9089 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.118234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea966e2-9bef-49ef-9984-e60db39b9089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.120996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea966e2-9bef-49ef-9984-e60db39b9089 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.223201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 967/1960] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:38 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:00:38.226458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebbd25-4e12-41de-8674-62d19f16b2ae None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.238800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ebbd25-4e12-41de-8674-62d19f16b2ae None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.297056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04247b37-50c4-4e48-9191-94d3cacc0f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.297755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04247b37-50c4-4e48-9191-94d3cacc0f8b admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.298072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04247b37-50c4-4e48-9191-94d3cacc0f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.298353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04247b37-50c4-4e48-9191-94d3cacc0f8b admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.299092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04247b37-50c4-4e48-9191-94d3cacc0f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.301082 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04247b37-50c4-4e48-9191-94d3cacc0f8b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.308218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:38.396783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 994/1961] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:38 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:38.401935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea966e2-9bef-49ef-9984-e60db39b9089 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.412168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea966e2-9bef-49ef-9984-e60db39b9089 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.417401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7149e-d40d-4aa8-9f48-3f296790b6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.418067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7149e-d40d-4aa8-9f48-3f296790b6bf admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.418354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7149e-d40d-4aa8-9f48-3f296790b6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.418635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7149e-d40d-4aa8-9f48-3f296790b6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.419363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7149e-d40d-4aa8-9f48-3f296790b6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.420991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb7149e-d40d-4aa8-9f48-3f296790b6bf admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.431390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 968/1962] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:38.435411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04247b37-50c4-4e48-9191-94d3cacc0f8b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.445820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04247b37-50c4-4e48-9191-94d3cacc0f8b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.450794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843fe06-9c21-47e4-b907-09e24c2ae2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.451472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843fe06-9c21-47e4-b907-09e24c2ae2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.451761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843fe06-9c21-47e4-b907-09e24c2ae2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.452044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843fe06-9c21-47e4-b907-09e24c2ae2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.452821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843fe06-9c21-47e4-b907-09e24c2ae2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.454456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843fe06-9c21-47e4-b907-09e24c2ae2c2 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.465776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 995/1963] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:38.470028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7149e-d40d-4aa8-9f48-3f296790b6bf admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.481026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7149e-d40d-4aa8-9f48-3f296790b6bf admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.486633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0766f-34d0-450e-8d73-a2438e97baec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.487554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0766f-34d0-450e-8d73-a2438e97baec admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.487837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0766f-34d0-450e-8d73-a2438e97baec admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.488134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0766f-34d0-450e-8d73-a2438e97baec admin admin] PATH_INFO: `/v3/projects/a5ac99a1996b465594ad5b6817107a8d/users/7e1a357609ce43829359b96d1f66bced/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.503918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0766f-34d0-450e-8d73-a2438e97baec admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ac99a1996b465594ad5b6817107a8d, user_id=7e1a357609ce43829359b96d1f66bced, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.506601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0766f-34d0-450e-8d73-a2438e97baec admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.614255 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 969/1964] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:38 2022] PUT /identity/v3/projects/a5ac99a1996b465594ad5b6817107a8d/users/7e1a357609ce43829359b96d1f66bced/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:38.624696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab06c1-6ecb-4ce7-aca7-c6643bbbbe70 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.624992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab06c1-6ecb-4ce7-aca7-c6643bbbbe70 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.625278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab06c1-6ecb-4ce7-aca7-c6643bbbbe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.655016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-acab06c1-6ecb-4ce7-aca7-c6643bbbbe70 None None] MFA Rules not processed for user `7e1a357609ce43829359b96d1f66bced`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:38.656189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-acab06c1-6ecb-4ce7-aca7-c6643bbbbe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:38.733036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 996/1965] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:38.740871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0766f-34d0-450e-8d73-a2438e97baec admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.821380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0766f-34d0-450e-8d73-a2438e97baec admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.827444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de29f1-f97e-4aeb-a87a-bd3f035dc36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.828157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de29f1-f97e-4aeb-a87a-bd3f035dc36e admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.828449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de29f1-f97e-4aeb-a87a-bd3f035dc36e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.828752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de29f1-f97e-4aeb-a87a-bd3f035dc36e admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.829448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de29f1-f97e-4aeb-a87a-bd3f035dc36e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.831361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de29f1-f97e-4aeb-a87a-bd3f035dc36e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:38.895472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 970/1966] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:38 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:38.899186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab06c1-6ecb-4ce7-aca7-c6643bbbbe70 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:38.910803 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab06c1-6ecb-4ce7-aca7-c6643bbbbe70 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:38.969935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85a2de-4dbc-4977-b136-e0939eb5c43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:38.970737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85a2de-4dbc-4977-b136-e0939eb5c43d admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:38.971093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85a2de-4dbc-4977-b136-e0939eb5c43d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:38.971469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85a2de-4dbc-4977-b136-e0939eb5c43d admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:38.972170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85a2de-4dbc-4977-b136-e0939eb5c43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:38.974146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85a2de-4dbc-4977-b136-e0939eb5c43d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:39.080465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 997/1967] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:38 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:39.081913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de29f1-f97e-4aeb-a87a-bd3f035dc36e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:39.092581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de29f1-f97e-4aeb-a87a-bd3f035dc36e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:39.097315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7364bcc-eb96-41b2-ba78-dbf2b9995a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:39.097979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7364bcc-eb96-41b2-ba78-dbf2b9995a64 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:39.098262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7364bcc-eb96-41b2-ba78-dbf2b9995a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:39.098536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7364bcc-eb96-41b2-ba78-dbf2b9995a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:39.099296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7364bcc-eb96-41b2-ba78-dbf2b9995a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:39.100940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7364bcc-eb96-41b2-ba78-dbf2b9995a64 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:39.102378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:39.113116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 971/1968] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:39.118648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85a2de-4dbc-4977-b136-e0939eb5c43d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:39.129562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85a2de-4dbc-4977-b136-e0939eb5c43d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:39.135644 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c821451-058c-4dee-8576-8654bcb96836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:39.136655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c821451-058c-4dee-8576-8654bcb96836 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:39.141584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c821451-058c-4dee-8576-8654bcb96836 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:39.141584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c821451-058c-4dee-8576-8654bcb96836 admin admin] PATH_INFO: `/v3/projects/c3f1030efec343b5afdf90f586fb1a74/users/4c640a85d5e74fe6a6c05ca5bc0c5e6a/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:39.155139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c821451-058c-4dee-8576-8654bcb96836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f1030efec343b5afdf90f586fb1a74, user_id=4c640a85d5e74fe6a6c05ca5bc0c5e6a, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:39.157977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c821451-058c-4dee-8576-8654bcb96836 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:39.248496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 998/1969] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:39 2022] PUT /identity/v3/projects/c3f1030efec343b5afdf90f586fb1a74/users/4c640a85d5e74fe6a6c05ca5bc0c5e6a/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:39.252885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7364bcc-eb96-41b2-ba78-dbf2b9995a64 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:39.265291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7364bcc-eb96-41b2-ba78-dbf2b9995a64 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:39.326469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b37d0-c8b3-4377-b642-3d6b67f7f94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:39.327242 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b37d0-c8b3-4377-b642-3d6b67f7f94e admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:39.327596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b37d0-c8b3-4377-b642-3d6b67f7f94e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:39.327943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b37d0-c8b3-4377-b642-3d6b67f7f94e admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:39.328777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b37d0-c8b3-4377-b642-3d6b67f7f94e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:39.330475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b37d0-c8b3-4377-b642-3d6b67f7f94e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:39.343322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 972/1970] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:39 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:39.347433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c821451-058c-4dee-8576-8654bcb96836 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:39.358638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c821451-058c-4dee-8576-8654bcb96836 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:39.363409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:39.364572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:39.364929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:39.365278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] PATH_INFO: `/v3/projects/c3f1030efec343b5afdf90f586fb1a74/users/4c640a85d5e74fe6a6c05ca5bc0c5e6a/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:39.367438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f1030efec343b5afdf90f586fb1a74, user_id=4c640a85d5e74fe6a6c05ca5bc0c5e6a, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:39.370727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:39.382648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 999/1971] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:39 2022] PUT /identity/v3/projects/c3f1030efec343b5afdf90f586fb1a74/users/4c640a85d5e74fe6a6c05ca5bc0c5e6a/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:39.393725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b547a-51fd-4fd7-927b-5aae5c9bd8c0 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:39.394021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b547a-51fd-4fd7-927b-5aae5c9bd8c0 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:39.394308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b547a-51fd-4fd7-927b-5aae5c9bd8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:39.421198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4a9b547a-51fd-4fd7-927b-5aae5c9bd8c0 None None] MFA Rules not processed for user `4c640a85d5e74fe6a6c05ca5bc0c5e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:39.422417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4a9b547a-51fd-4fd7-927b-5aae5c9bd8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:39.490650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 973/1972] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:39.501913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:39.503611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:39.529999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:39.568727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee403bc-50cf-4ed1-8a02-b03533ef6847 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:39.572388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:39.573029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:39.573313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:39.573590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:39.582564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:39.584061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:39.594629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1000/1973] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:40.921474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b547a-51fd-4fd7-927b-5aae5c9bd8c0 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:40.931448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b547a-51fd-4fd7-927b-5aae5c9bd8c0 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:40.935258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbb1fb-69a7-4a99-9d9c-9d7384e8fb19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:40.935914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbb1fb-69a7-4a99-9d9c-9d7384e8fb19 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:40.936227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbb1fb-69a7-4a99-9d9c-9d7384e8fb19 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:40.936509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbb1fb-69a7-4a99-9d9c-9d7384e8fb19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:40.994058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbb1fb-69a7-4a99-9d9c-9d7384e8fb19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:40.995446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbb1fb-69a7-4a99-9d9c-9d7384e8fb19 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:41.004383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 974/1974] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:45.427152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:45.428688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:45.496796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249bdd4-0867-49be-9384-bc5f099db3f5 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:45.501390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:45.502295 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.502579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.502857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] PATH_INFO: `/v3/users/c0552145127647279abaff5f1f347fe9` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.504046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0552145127647279abaff5f1f347fe9)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:45.505992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:45.689847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0552145127647279abaff5f1f347fe9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:45.761627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0552145127647279abaff5f1f347fe9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:45.762506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0552145127647279abaff5f1f347fe9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:45.783436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0552145127647279abaff5f1f347fe9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:45.784456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0552145127647279abaff5f1f347fe9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:45.803622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1001/1975] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:45 2022] DELETE /identity/v3/users/c0552145127647279abaff5f1f347fe9 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:45.808657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69875-c9b9-4d46-b872-54de0d7cb855 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.809129 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69875-c9b9-4d46-b872-54de0d7cb855 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.809296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69875-c9b9-4d46-b872-54de0d7cb855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.828690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-19f69875-c9b9-4d46-b872-54de0d7cb855 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:45.835957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-19f69875-c9b9-4d46-b872-54de0d7cb855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:45.841315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:45.894195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 975/1976] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:45.904665 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:45.906230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:45.952768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:45.969277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf92a4d-7829-4452-9be4-82a1460095d7 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:45.971724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:45.972443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:45.972742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:45.973029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:45.976665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f69875-c9b9-4d46-b872-54de0d7cb855 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:45.978060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-19f69875-c9b9-4d46-b872-54de0d7cb855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:45.984778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:45.986228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:45.996571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1002/1977] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:46.049506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f69875-c9b9-4d46-b872-54de0d7cb855 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.053620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a97ce-3754-4afd-9faa-ed53021f22c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.054457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a97ce-3754-4afd-9faa-ed53021f22c9 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.054748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a97ce-3754-4afd-9faa-ed53021f22c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.055034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a97ce-3754-4afd-9faa-ed53021f22c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.057111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f60a97ce-3754-4afd-9faa-ed53021f22c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:46.147259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a97ce-3754-4afd-9faa-ed53021f22c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.149142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a97ce-3754-4afd-9faa-ed53021f22c9 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.161381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 976/1978] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:46.613301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.615610 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:46.631761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a73e1-9741-474d-a477-585157d22b8f service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.639501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:46.646752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.646752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.646752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] PATH_INFO: `/v3/projects/3077c72cf2874f58bf91311d581450e9` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.646752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3077c72cf2874f58bf91311d581450e9)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:46.646752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:46.775233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3077c72cf2874f58bf91311d581450e9', 'request_id': 'req-77daa5ed-d64b-4a55-a270-0de2974b8309'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:46.835075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3077c72cf2874f58bf91311d581450e9', 'request_id': 'req-77daa5ed-d64b-4a55-a270-0de2974b8309'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:46.841817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3077c72cf2874f58bf91311d581450e9', 'request_id': 'req-77daa5ed-d64b-4a55-a270-0de2974b8309'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:46.853002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] The token cache is being invalidate because project 3077c72cf2874f58bf91311d581450e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:46.853303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:46.854154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-77daa5ed-d64b-4a55-a270-0de2974b8309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:46.855726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1003/1979] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:46 2022] DELETE /identity/v3/projects/3077c72cf2874f58bf91311d581450e9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:46.860611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0039e-5e22-4e6b-a7c3-0452db3a4b1b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.863736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0039e-5e22-4e6b-a7c3-0452db3a4b1b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.863736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0039e-5e22-4e6b-a7c3-0452db3a4b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.880107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ffc0039e-5e22-4e6b-a7c3-0452db3a4b1b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:46.886064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ffc0039e-5e22-4e6b-a7c3-0452db3a4b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:46.942500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 977/1980] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:46.950923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21f0d-e6f8-41e2-8761-1ffe8c908f35 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:46.951214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21f0d-e6f8-41e2-8761-1ffe8c908f35 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:46.951497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b21f0d-e6f8-41e2-8761-1ffe8c908f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:46.970606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-21b21f0d-e6f8-41e2-8761-1ffe8c908f35 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:46.971797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-21b21f0d-e6f8-41e2-8761-1ffe8c908f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:46.980291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1004/1981] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:46.984611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0039e-5e22-4e6b-a7c3-0452db3a4b1b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:46.994637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0039e-5e22-4e6b-a7c3-0452db3a4b1b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:46.999451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.000316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.000609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.000886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.001618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:47.001926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.003604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.011375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:47.016404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 978/1982] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:00:47.020746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b21f0d-e6f8-41e2-8761-1ffe8c908f35 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.030843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b21f0d-e6f8-41e2-8761-1ffe8c908f35 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.035491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3356b8-5a20-447d-81bf-ad236aa2fc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.036171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3356b8-5a20-447d-81bf-ad236aa2fc67 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.036457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3356b8-5a20-447d-81bf-ad236aa2fc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.036734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3356b8-5a20-447d-81bf-ad236aa2fc67 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.037430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3356b8-5a20-447d-81bf-ad236aa2fc67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.039349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3356b8-5a20-447d-81bf-ad236aa2fc67 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.166688 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1005/1983] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:47 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:47.168821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.179667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd756c-ef91-4e1c-aafd-35a6f0c0a750 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.237911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3c42f-924c-47a0-bffd-3a63359d91e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.238602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3c42f-924c-47a0-bffd-3a63359d91e1 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.238890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3c42f-924c-47a0-bffd-3a63359d91e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.239301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3c42f-924c-47a0-bffd-3a63359d91e1 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.240027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3c42f-924c-47a0-bffd-3a63359d91e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.242134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3c42f-924c-47a0-bffd-3a63359d91e1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.315265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 979/1984] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:47 2022] POST /identity/v3/users => generated 288 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:47.316537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3356b8-5a20-447d-81bf-ad236aa2fc67 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.327447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3356b8-5a20-447d-81bf-ad236aa2fc67 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.332182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdec25-656f-470f-a43b-c1a6e3098e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.332859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdec25-656f-470f-a43b-c1a6e3098e09 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.333440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdec25-656f-470f-a43b-c1a6e3098e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.333440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdec25-656f-470f-a43b-c1a6e3098e09 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.334141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdec25-656f-470f-a43b-c1a6e3098e09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.335721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdec25-656f-470f-a43b-c1a6e3098e09 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.357584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1006/1985] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:47.364327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3c42f-924c-47a0-bffd-3a63359d91e1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.375503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3c42f-924c-47a0-bffd-3a63359d91e1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.380395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a16f48-8160-4b7a-ad2a-b1303d511a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.381067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a16f48-8160-4b7a-ad2a-b1303d511a02 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.381355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a16f48-8160-4b7a-ad2a-b1303d511a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.381634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a16f48-8160-4b7a-ad2a-b1303d511a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.382404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a16f48-8160-4b7a-ad2a-b1303d511a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.384019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a16f48-8160-4b7a-ad2a-b1303d511a02 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.395339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 980/1986] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:47.399682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdec25-656f-470f-a43b-c1a6e3098e09 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.410661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdec25-656f-470f-a43b-c1a6e3098e09 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.415781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.418717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.418717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.418717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] PATH_INFO: `/v3/projects/c73436901fc24d3cac76e4110b7ae237/users/2cb62ed0a9604c5fbd24196709e1e1f8/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.434913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c73436901fc24d3cac76e4110b7ae237, user_id=2cb62ed0a9604c5fbd24196709e1e1f8, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.436718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.550555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1007/1987] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:47 2022] PUT /identity/v3/projects/c73436901fc24d3cac76e4110b7ae237/users/2cb62ed0a9604c5fbd24196709e1e1f8/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:47.562756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36c88a-7717-44fc-bfc4-917d078e3edd None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.563053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36c88a-7717-44fc-bfc4-917d078e3edd None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.563335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b36c88a-7717-44fc-bfc4-917d078e3edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.592762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7b36c88a-7717-44fc-bfc4-917d078e3edd None None] MFA Rules not processed for user `2cb62ed0a9604c5fbd24196709e1e1f8`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:47.594145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7b36c88a-7717-44fc-bfc4-917d078e3edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:47.681587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 981/1988] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:47.707829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:47.709728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:47.775081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0874294-ca29-4710-804d-7a3103ec8199 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:47.778980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bf343-313a-4028-977e-b156bba52428 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:47.779634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bf343-313a-4028-977e-b156bba52428 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:47.779935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bf343-313a-4028-977e-b156bba52428 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:47.780242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bf343-313a-4028-977e-b156bba52428 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:47.789577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bf343-313a-4028-977e-b156bba52428 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:47.791566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bf343-313a-4028-977e-b156bba52428 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:47.801175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1008/1989] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:50.396363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36c88a-7717-44fc-bfc4-917d078e3edd None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:50.397842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7b36c88a-7717-44fc-bfc4-917d078e3edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:50.479988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b36c88a-7717-44fc-bfc4-917d078e3edd None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:50.484967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:50.485970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:50.486205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:50.486488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] PATH_INFO: `/v3/users/4394dbebd4e34164add908888891a143` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:50.487738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4394dbebd4e34164add908888891a143)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:50.489783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:50.497739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:50.666596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4394dbebd4e34164add908888891a143'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:50.732485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4394dbebd4e34164add908888891a143'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:50.733112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4394dbebd4e34164add908888891a143'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:50.744949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4394dbebd4e34164add908888891a143'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:50.744949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4394dbebd4e34164add908888891a143'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:50.778653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 982/1990] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:50 2022] DELETE /identity/v3/users/4394dbebd4e34164add908888891a143 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:50.783530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dee6e-96c1-4f04-9c08-df69a6597c8b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:50.783999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dee6e-96c1-4f04-9c08-df69a6597c8b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:50.784427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dee6e-96c1-4f04-9c08-df69a6597c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:50.797723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:50.803769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-7c9dee6e-96c1-4f04-9c08-df69a6597c8b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:50.805080 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7c9dee6e-96c1-4f04-9c08-df69a6597c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:50.876630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1009/1991] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:50.884196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:50.889539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:50.954102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d301c8-b991-44f5-a5be-c1f4c33ab000 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:50.958199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4acb18-cafd-4282-bf0b-df022702673e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:50.958861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4acb18-cafd-4282-bf0b-df022702673e service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:50.959151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4acb18-cafd-4282-bf0b-df022702673e service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:50.959478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4acb18-cafd-4282-bf0b-df022702673e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:50.970783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4acb18-cafd-4282-bf0b-df022702673e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:50.975815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4acb18-cafd-4282-bf0b-df022702673e service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:50.983427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 983/1992] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:51.789096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dee6e-96c1-4f04-9c08-df69a6597c8b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:51.791123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7c9dee6e-96c1-4f04-9c08-df69a6597c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:51.804494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dee6e-96c1-4f04-9c08-df69a6597c8b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:51.811442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:51.812904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:51.813337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:51.813761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] PATH_INFO: `/v3/projects/5429691aa9c04c97b3c68d9a4df13659` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:51.815539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5429691aa9c04c97b3c68d9a4df13659)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:51.818915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:51.920103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5429691aa9c04c97b3c68d9a4df13659', 'request_id': 'req-41f5bd30-cf05-4073-9cea-f92ac7402c2a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:51.990152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5429691aa9c04c97b3c68d9a4df13659', 'request_id': 'req-41f5bd30-cf05-4073-9cea-f92ac7402c2a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.002888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5429691aa9c04c97b3c68d9a4df13659', 'request_id': 'req-41f5bd30-cf05-4073-9cea-f92ac7402c2a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.011262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] The token cache is being invalidate because project 5429691aa9c04c97b3c68d9a4df13659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:52.011808 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.013111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.015427 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1010/1993] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:51 2022] DELETE /identity/v3/projects/5429691aa9c04c97b3c68d9a4df13659 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:52.019380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4acb18-cafd-4282-bf0b-df022702673e service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:52.043224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4acb18-cafd-4282-bf0b-df022702673e service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:52.107801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:52.108806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:52.109140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:52.109441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] PATH_INFO: `/v3/users/7e1a357609ce43829359b96d1f66bced` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:52.110652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e1a357609ce43829359b96d1f66bced)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:52.112637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:52.288007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:52.303914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e1a357609ce43829359b96d1f66bced'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.376697 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e1a357609ce43829359b96d1f66bced'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.377311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e1a357609ce43829359b96d1f66bced'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.385600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e1a357609ce43829359b96d1f66bced'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.386550 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e1a357609ce43829359b96d1f66bced'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:52.406500 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 984/1994] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:52 2022] DELETE /identity/v3/users/7e1a357609ce43829359b96d1f66bced => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:52.969185 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:52.970650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:52.985978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5bd30-cf05-4073-9cea-f92ac7402c2a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:53.073061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:53.073061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:53.073956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:53.074258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] PATH_INFO: `/v3/projects/a5ac99a1996b465594ad5b6817107a8d` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:53.075636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ac99a1996b465594ad5b6817107a8d)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:53.089866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:53.137049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:53.139747 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:53.154110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818f66e-ea7e-4c61-830c-c10f913dc156 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:53.160656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:53.162233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:53.162796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:53.163204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] PATH_INFO: `/v3/users/9078f21cce334aea96c3e8339329bce4` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:53.164759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9078f21cce334aea96c3e8339329bce4)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:53.168087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:53.204701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:53.231190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ac99a1996b465594ad5b6817107a8d', 'request_id': 'req-0c0adae2-d376-4bcb-898d-82a3817036a0'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.325007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ac99a1996b465594ad5b6817107a8d', 'request_id': 'req-0c0adae2-d376-4bcb-898d-82a3817036a0'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.331104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ac99a1996b465594ad5b6817107a8d', 'request_id': 'req-0c0adae2-d376-4bcb-898d-82a3817036a0'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.343096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] The token cache is being invalidate because project a5ac99a1996b465594ad5b6817107a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:53.343455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.344023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.345617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1011/1995] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:52 2022] DELETE /identity/v3/projects/a5ac99a1996b465594ad5b6817107a8d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:53.349454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:53.350814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:53.376427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9078f21cce334aea96c3e8339329bce4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.427695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0adae2-d376-4bcb-898d-82a3817036a0 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:53.436085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:53.437107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:53.437423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:53.437699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] PATH_INFO: `/v3/users/4c640a85d5e74fe6a6c05ca5bc0c5e6a` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:53.439042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c640a85d5e74fe6a6c05ca5bc0c5e6a)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:53.441018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:53.474977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9078f21cce334aea96c3e8339329bce4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.475690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9078f21cce334aea96c3e8339329bce4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.486005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9078f21cce334aea96c3e8339329bce4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.486857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-bf6160e0-4d0c-4f8c-b001-f9fd1dd5026a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9078f21cce334aea96c3e8339329bce4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.510857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 985/1996] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:53 2022] DELETE /identity/v3/users/9078f21cce334aea96c3e8339329bce4 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:53.519303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9384f-cdf9-4036-bfd1-85f2e11ab6a9 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:53.519303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9384f-cdf9-4036-bfd1-85f2e11ab6a9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:53.519303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9384f-cdf9-4036-bfd1-85f2e11ab6a9 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:53.519303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 986/1997] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 06:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:00:53.519303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6272a-66b1-4b15-b6e7-36665f14c415 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:53.519303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6272a-66b1-4b15-b6e7-36665f14c415 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:53.519838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6272a-66b1-4b15-b6e7-36665f14c415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:53.534961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-adb6272a-66b1-4b15-b6e7-36665f14c415 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:53.536256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-adb6272a-66b1-4b15-b6e7-36665f14c415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:53.605163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 987/1998] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:53.607433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9fab3-6add-4beb-b468-e5728b20b8d2 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:53.607836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9fab3-6add-4beb-b468-e5728b20b8d2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:53.612515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9fab3-6add-4beb-b468-e5728b20b8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:53.636151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-27b9fab3-6add-4beb-b468-e5728b20b8d2 None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:53.637411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-27b9fab3-6add-4beb-b468-e5728b20b8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:53.655292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:53.674652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c640a85d5e74fe6a6c05ca5bc0c5e6a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.726474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 988/1999] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 06:00:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:53.737802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9fab3-6add-4beb-b468-e5728b20b8d2 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:53.737802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-27b9fab3-6add-4beb-b468-e5728b20b8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:53.758512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c640a85d5e74fe6a6c05ca5bc0c5e6a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.760613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c640a85d5e74fe6a6c05ca5bc0c5e6a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.776051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c640a85d5e74fe6a6c05ca5bc0c5e6a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.776051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c640a85d5e74fe6a6c05ca5bc0c5e6a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:53.801524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1012/2000] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:53 2022] DELETE /identity/v3/users/4c640a85d5e74fe6a6c05ca5bc0c5e6a => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:53.802535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:53.804021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:53.816144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9fab3-6add-4beb-b468-e5728b20b8d2 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:53.869860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:53.906963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5d20a-8174-49ed-aa5d-5b560c0c993e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:53.912264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:53.914153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:53.914369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:53.914626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:53.914968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:53.916666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:53.917310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:53.917773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] PATH_INFO: `/v3/users/db1a33dc0f174cb1bdebf9e0c710509d` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:53.919125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:53.919816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1a33dc0f174cb1bdebf9e0c710509d)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:53.922345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:53.935821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:53.937497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:53.950010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 989/2001] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:53.953640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:53.956248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:54.041451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681cc12d-3dee-4275-bcbd-be69713b3046 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:54.047524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:54.048235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.048523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.048805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.051674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:54.066706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:54.068790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:54.083461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 990/2002] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:54.137085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1a33dc0f174cb1bdebf9e0c710509d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.215510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1a33dc0f174cb1bdebf9e0c710509d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.215510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1a33dc0f174cb1bdebf9e0c710509d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.226515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1a33dc0f174cb1bdebf9e0c710509d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.227563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1a33dc0f174cb1bdebf9e0c710509d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.273303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1013/2003] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:53 2022] DELETE /identity/v3/users/db1a33dc0f174cb1bdebf9e0c710509d => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:54.368292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:54.371374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:54.381766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:54.387894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b5e8-a6bc-4229-af51-d7d8365d20cb service placement] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:54.477709 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:54.478295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.481300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.481300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] PATH_INFO: `/v3/projects/c3f1030efec343b5afdf90f586fb1a74` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.482309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f1030efec343b5afdf90f586fb1a74)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:54.485409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:54.611679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:54.613325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:54.631107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb77835c-da52-4fb1-a2f6-a0f41215ae40 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:54.637204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f1030efec343b5afdf90f586fb1a74', 'request_id': 'req-10cb853f-ead3-48af-a71b-e33b0fa28422'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.719153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:54.720183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.720540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.720886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] PATH_INFO: `/v3/projects/b21e65c1c53f48c39dcf39b8d05a71be` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.722148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21e65c1c53f48c39dcf39b8d05a71be)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:54.724268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:54.738836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f1030efec343b5afdf90f586fb1a74', 'request_id': 'req-10cb853f-ead3-48af-a71b-e33b0fa28422'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.746031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f1030efec343b5afdf90f586fb1a74', 'request_id': 'req-10cb853f-ead3-48af-a71b-e33b0fa28422'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.765566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] The token cache is being invalidate because project c3f1030efec343b5afdf90f586fb1a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:54.765863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.768859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.779294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 991/2004] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:54 2022] DELETE /identity/v3/projects/c3f1030efec343b5afdf90f586fb1a74 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:54.780251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:54.784429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:54.799397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb853f-ead3-48af-a71b-e33b0fa28422 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:54.804617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:54.805571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.805929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.806883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] PATH_INFO: `/v3/projects/1484746571a24f5db36ed027469f88c1` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.808201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1484746571a24f5db36ed027469f88c1)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:54.810419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:54.854859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21e65c1c53f48c39dcf39b8d05a71be', 'request_id': 'req-d24af2ca-6765-4c94-9c4c-ffdabd25d663'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.926434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21e65c1c53f48c39dcf39b8d05a71be', 'request_id': 'req-d24af2ca-6765-4c94-9c4c-ffdabd25d663'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.941221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21e65c1c53f48c39dcf39b8d05a71be', 'request_id': 'req-d24af2ca-6765-4c94-9c4c-ffdabd25d663'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.953465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] The token cache is being invalidate because project b21e65c1c53f48c39dcf39b8d05a71be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:54.953465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.953465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d24af2ca-6765-4c94-9c4c-ffdabd25d663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.953465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1014/2005] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:54 2022] DELETE /identity/v3/projects/b21e65c1c53f48c39dcf39b8d05a71be => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:00:54.953465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ff77c-1665-446b-9ae1-a6d1a543a114 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:54.954501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ff77c-1665-446b-9ae1-a6d1a543a114 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:54.954501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ff77c-1665-446b-9ae1-a6d1a543a114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:54.958997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1484746571a24f5db36ed027469f88c1', 'request_id': 'req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:54.976947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ec3ff77c-1665-446b-9ae1-a6d1a543a114 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:54.980105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ec3ff77c-1665-446b-9ae1-a6d1a543a114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:55.001125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:55.024290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1484746571a24f5db36ed027469f88c1', 'request_id': 'req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:55.031403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1484746571a24f5db36ed027469f88c1', 'request_id': 'req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:55.045225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] The token cache is being invalidate because project 1484746571a24f5db36ed027469f88c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:00:55.045610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:55.046421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5e76f8e8-3639-4ba3-b8a1-10634f500c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:00:55.047944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1015/2006] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:55.048553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 992/2007] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:00:54 2022] DELETE /identity/v3/projects/1484746571a24f5db36ed027469f88c1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:55.049287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89801bb1-fb4b-4158-ba55-2cabfceb5350 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.049579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89801bb1-fb4b-4158-ba55-2cabfceb5350 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.049857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89801bb1-fb4b-4158-ba55-2cabfceb5350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.053553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4507c-dcad-4fc0-a6c1-836c291b64c7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.053852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4507c-dcad-4fc0-a6c1-836c291b64c7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.054234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4507c-dcad-4fc0-a6c1-836c291b64c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.070326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-89801bb1-fb4b-4158-ba55-2cabfceb5350 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:55.071614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-89801bb1-fb4b-4158-ba55-2cabfceb5350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:55.074546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-13a4507c-dcad-4fc0-a6c1-836c291b64c7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:55.075912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-13a4507c-dcad-4fc0-a6c1-836c291b64c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:55.080929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1016/2008] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:55.085484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67c97-5690-4c6d-9ae9-6892ca220e82 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.085829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67c97-5690-4c6d-9ae9-6892ca220e82 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.086139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa67c97-5690-4c6d-9ae9-6892ca220e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.096292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 993/2009] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:55.096292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c48c0-9778-4493-aee2-161ccfa7f733 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.096292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c48c0-9778-4493-aee2-161ccfa7f733 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.096292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c48c0-9778-4493-aee2-161ccfa7f733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.111852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2aa67c97-5690-4c6d-9ae9-6892ca220e82 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:55.113199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2aa67c97-5690-4c6d-9ae9-6892ca220e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:55.116462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e79c48c0-9778-4493-aee2-161ccfa7f733 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:55.117636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e79c48c0-9778-4493-aee2-161ccfa7f733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:55.122007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1017/2010] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:55.123568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fe2af-ef3e-4f8c-b90f-13abae0ec917 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.124001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fe2af-ef3e-4f8c-b90f-13abae0ec917 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.124397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fe2af-ef3e-4f8c-b90f-13abae0ec917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.131525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 994/2011] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:55.132534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c48c0-9778-4493-aee2-161ccfa7f733 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.142063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:55.144776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2a0fe2af-ef3e-4f8c-b90f-13abae0ec917 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:55.146059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2a0fe2af-ef3e-4f8c-b90f-13abae0ec917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:55.146331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c48c0-9778-4493-aee2-161ccfa7f733 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.153657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.154453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.154755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.155104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.157069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:55.157069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.157267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1018/2012] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:55.158045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fe2af-ef3e-4f8c-b90f-13abae0ec917 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.158430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.168301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fe2af-ef3e-4f8c-b90f-13abae0ec917 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.173614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.174260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 995/2013] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:55.174576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.174899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.175211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.175961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:55.176304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.176592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.177898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.187254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea78c1-8d6e-4131-bb40-d2a8d8938406 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.191593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1019/2014] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:55.192587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.196350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.197995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.198466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.198778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.199704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:00:55.200087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.201943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.205091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be50a8-adbc-48bd-9d3f-3c8ae7703e8b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.209854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea2a62-0a87-4263-81ac-0c8a4afa90e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.210519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea2a62-0a87-4263-81ac-0c8a4afa90e9 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.210944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea2a62-0a87-4263-81ac-0c8a4afa90e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.211352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea2a62-0a87-4263-81ac-0c8a4afa90e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.212050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea2a62-0a87-4263-81ac-0c8a4afa90e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.214022 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea2a62-0a87-4263-81ac-0c8a4afa90e9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.215832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 996/2015] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:00:55.216641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.226249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af8942e-ac9c-4c05-961e-59b804c2f0f0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.237660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4f2ac-86a6-48a8-b630-d06679e85582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.238451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4f2ac-86a6-48a8-b630-d06679e85582 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.238735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4f2ac-86a6-48a8-b630-d06679e85582 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.239017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb4f2ac-86a6-48a8-b630-d06679e85582 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.239717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4f2ac-86a6-48a8-b630-d06679e85582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.241707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb4f2ac-86a6-48a8-b630-d06679e85582 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.297529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1020/2016] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:55.297529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea2a62-0a87-4263-81ac-0c8a4afa90e9 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.360234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea2a62-0a87-4263-81ac-0c8a4afa90e9 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.365863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 997/2017] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:55.365863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4f2ac-86a6-48a8-b630-d06679e85582 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.425478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77face90-b734-4ec6-9ee0-aea70b7c1d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.426151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77face90-b734-4ec6-9ee0-aea70b7c1d2b admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.426448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77face90-b734-4ec6-9ee0-aea70b7c1d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.426729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77face90-b734-4ec6-9ee0-aea70b7c1d2b admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.427419 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77face90-b734-4ec6-9ee0-aea70b7c1d2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.429451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77face90-b734-4ec6-9ee0-aea70b7c1d2b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.438153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb4f2ac-86a6-48a8-b630-d06679e85582 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.444599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38678dd-6604-47b0-9d26-5f2bd55d0e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.445291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38678dd-6604-47b0-9d26-5f2bd55d0e4f admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.445580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38678dd-6604-47b0-9d26-5f2bd55d0e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.445871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38678dd-6604-47b0-9d26-5f2bd55d0e4f admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.446704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38678dd-6604-47b0-9d26-5f2bd55d0e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.448814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38678dd-6604-47b0-9d26-5f2bd55d0e4f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.522411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1021/2018] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/projects => generated 367 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:55.526261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77face90-b734-4ec6-9ee0-aea70b7c1d2b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.547140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:55.553439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 998/2019] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:55.553439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38678dd-6604-47b0-9d26-5f2bd55d0e4f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.564649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38678dd-6604-47b0-9d26-5f2bd55d0e4f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.601202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77face90-b734-4ec6-9ee0-aea70b7c1d2b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.607525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0c4e-338e-48fb-b5ee-dc6924b4a3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.608232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0c4e-338e-48fb-b5ee-dc6924b4a3bf admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.608525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0c4e-338e-48fb-b5ee-dc6924b4a3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.608804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a0c4e-338e-48fb-b5ee-dc6924b4a3bf admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.609534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0c4e-338e-48fb-b5ee-dc6924b4a3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.612077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a0c4e-338e-48fb-b5ee-dc6924b4a3bf admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.625595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329a537-761f-48da-87bf-c51d4a80d9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.628029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329a537-761f-48da-87bf-c51d4a80d9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.628029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329a537-761f-48da-87bf-c51d4a80d9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.628029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329a537-761f-48da-87bf-c51d4a80d9a3 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.628029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329a537-761f-48da-87bf-c51d4a80d9a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.629771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329a537-761f-48da-87bf-c51d4a80d9a3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.726646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1022/2020] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:55.726848 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 999/2021] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:00:55.727763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329a537-761f-48da-87bf-c51d4a80d9a3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.732237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0c4e-338e-48fb-b5ee-dc6924b4a3bf admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.739196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329a537-761f-48da-87bf-c51d4a80d9a3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.742669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a0c4e-338e-48fb-b5ee-dc6924b4a3bf admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.744823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf601f8-037c-42e1-9842-f10a96c8b80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.745486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf601f8-037c-42e1-9842-f10a96c8b80b admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.745768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf601f8-037c-42e1-9842-f10a96c8b80b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.746043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf601f8-037c-42e1-9842-f10a96c8b80b admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.746770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf601f8-037c-42e1-9842-f10a96c8b80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.748355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09219fb-7b14-460b-b3a0-0863b61ca93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.753264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf601f8-037c-42e1-9842-f10a96c8b80b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.753386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09219fb-7b14-460b-b3a0-0863b61ca93f admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.753386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09219fb-7b14-460b-b3a0-0863b61ca93f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.753386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09219fb-7b14-460b-b3a0-0863b61ca93f admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.753386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09219fb-7b14-460b-b3a0-0863b61ca93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.753386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09219fb-7b14-460b-b3a0-0863b61ca93f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.766823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1023/2022] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:00:55.766823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09219fb-7b14-460b-b3a0-0863b61ca93f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.774103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1000/2023] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:55.782016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf601f8-037c-42e1-9842-f10a96c8b80b admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.782132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09219fb-7b14-460b-b3a0-0863b61ca93f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.782132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830a597-68a5-4067-980f-0fd61302c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.783352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830a597-68a5-4067-980f-0fd61302c525 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.783650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830a597-68a5-4067-980f-0fd61302c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.783934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830a597-68a5-4067-980f-0fd61302c525 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.784913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830a597-68a5-4067-980f-0fd61302c525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.786455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf601f8-037c-42e1-9842-f10a96c8b80b admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.786584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830a597-68a5-4067-980f-0fd61302c525 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.792105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee6e13-b916-4321-bc46-2ea41d33d719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.792925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee6e13-b916-4321-bc46-2ea41d33d719 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.793340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee6e13-b916-4321-bc46-2ea41d33d719 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.793751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee6e13-b916-4321-bc46-2ea41d33d719 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.794347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee6e13-b916-4321-bc46-2ea41d33d719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.795985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee6e13-b916-4321-bc46-2ea41d33d719 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.798776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1024/2024] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:55.799570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830a597-68a5-4067-980f-0fd61302c525 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.809681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1001/2025] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:55.810416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee6e13-b916-4321-bc46-2ea41d33d719 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.810523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830a597-68a5-4067-980f-0fd61302c525 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.815548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcdc95-a5d9-4c83-895b-c506c2e2a4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.816579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcdc95-a5d9-4c83-895b-c506c2e2a4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.817369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcdc95-a5d9-4c83-895b-c506c2e2a4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.817683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dcdc95-a5d9-4c83-895b-c506c2e2a4b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.818722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcdc95-a5d9-4c83-895b-c506c2e2a4b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.820389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dcdc95-a5d9-4c83-895b-c506c2e2a4b3 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.825963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee6e13-b916-4321-bc46-2ea41d33d719 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.832511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1025/2026] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:55.832647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f593c8-a3fb-4ec8-bcab-3573e74f70f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.832647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f593c8-a3fb-4ec8-bcab-3573e74f70f9 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.835397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcdc95-a5d9-4c83-895b-c506c2e2a4b3 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.835490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f593c8-a3fb-4ec8-bcab-3573e74f70f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.835490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f593c8-a3fb-4ec8-bcab-3573e74f70f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.835490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f593c8-a3fb-4ec8-bcab-3573e74f70f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.835490 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f593c8-a3fb-4ec8-bcab-3573e74f70f9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.842555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dcdc95-a5d9-4c83-895b-c506c2e2a4b3 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.848790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1002/2027] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:00:55.848790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f593c8-a3fb-4ec8-bcab-3573e74f70f9 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.849000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a11701-f80b-4058-8bbb-a923b4c41ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.849000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a11701-f80b-4058-8bbb-a923b4c41ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.849000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a11701-f80b-4058-8bbb-a923b4c41ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.849268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a11701-f80b-4058-8bbb-a923b4c41ae9 admin admin] PATH_INFO: `/v3/projects/7d97f354896a4f1bbef0b4d1e6b99c27/users/5caf24262d214d9aa8cf9612582154f5/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.851944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:55.857445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f593c8-a3fb-4ec8-bcab-3573e74f70f9 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:55.862420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2060d-2814-4f98-aa9d-217139a578eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:55.863573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2060d-2814-4f98-aa9d-217139a578eb admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.863933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2060d-2814-4f98-aa9d-217139a578eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.864298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2060d-2814-4f98-aa9d-217139a578eb admin admin] PATH_INFO: `/v3/projects/50d4a8f19ba9426095868160e165016b/users/fee17de0ec784ddda697e3ccc6cd1f65/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:55.866615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a11701-f80b-4058-8bbb-a923b4c41ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d97f354896a4f1bbef0b4d1e6b99c27, user_id=5caf24262d214d9aa8cf9612582154f5, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.869405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a11701-f80b-4058-8bbb-a923b4c41ae9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.879557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2060d-2814-4f98-aa9d-217139a578eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=50d4a8f19ba9426095868160e165016b, user_id=fee17de0ec784ddda697e3ccc6cd1f65, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:55.882317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2060d-2814-4f98-aa9d-217139a578eb admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:55.944085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1026/2028] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:55 2022] PUT /identity/v3/projects/7d97f354896a4f1bbef0b4d1e6b99c27/users/5caf24262d214d9aa8cf9612582154f5/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:55.945067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a11701-f80b-4058-8bbb-a923b4c41ae9 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:55.980764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1003/2029] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:55 2022] PUT /identity/v3/projects/50d4a8f19ba9426095868160e165016b/users/fee17de0ec784ddda697e3ccc6cd1f65/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:55.982087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47078-4259-4419-a02d-0a09b7237788 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:55.982314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47078-4259-4419-a02d-0a09b7237788 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:55.982571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47078-4259-4419-a02d-0a09b7237788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.000435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b9a47078-4259-4419-a02d-0a09b7237788 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:56.001662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b9a47078-4259-4419-a02d-0a09b7237788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:56.016720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a11701-f80b-4058-8bbb-a923b4c41ae9 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:56.020542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:56.078908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:56.080252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:56.080542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:56.080818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] PATH_INFO: `/v3/projects/fe6bc36a72b14577ad900d4ba4e1c382/users/96ea34f608184b31bdb0f155bab10301/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.082801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1004/2030] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:56.084245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f174cb-d818-4bc9-ac75-d1d73d11d044 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:56.084547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f174cb-d818-4bc9-ac75-d1d73d11d044 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:56.084829 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f174cb-d818-4bc9-ac75-d1d73d11d044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.097601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6bc36a72b14577ad900d4ba4e1c382, user_id=96ea34f608184b31bdb0f155bab10301, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:56.100443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:56.115093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a3f174cb-d818-4bc9-ac75-d1d73d11d044 None None] MFA Rules not processed for user `fee17de0ec784ddda697e3ccc6cd1f65`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:56.116767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a3f174cb-d818-4bc9-ac75-d1d73d11d044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:56.178758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1027/2031] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:00:55 2022] PUT /identity/v3/projects/fe6bc36a72b14577ad900d4ba4e1c382/users/96ea34f608184b31bdb0f155bab10301/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:00:56.189173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:56.189173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:56.189382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1005/2032] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:56.190240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c1a06-d3a0-4698-be65-5b31ffb50c04 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:56.190771 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c1a06-d3a0-4698-be65-5b31ffb50c04 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:56.191301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c1a06-d3a0-4698-be65-5b31ffb50c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.221882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b96c1a06-d3a0-4698-be65-5b31ffb50c04 None None] MFA Rules not processed for user `96ea34f608184b31bdb0f155bab10301`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:56.223127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b96c1a06-d3a0-4698-be65-5b31ffb50c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:56.241935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a7773-830d-4229-b5ea-7a1ddcdba412 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:56.245997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:56.246650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:56.246934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:56.247211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.286589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:56.292143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1006/2033] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:00:56.292994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c1a06-d3a0-4698-be65-5b31ffb50c04 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:56.294297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b96c1a06-d3a0-4698-be65-5b31ffb50c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:56.317058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:56.318429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:56.329016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1028/2034] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:56.338099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:56.340531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:56.360703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c1a06-d3a0-4698-be65-5b31ffb50c04 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:56.364419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4761-810d-48b4-bd14-b7e6600057f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:56.365073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4761-810d-48b4-bd14-b7e6600057f7 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:56.365354 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4761-810d-48b4-bd14-b7e6600057f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:56.365626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d4761-810d-48b4-bd14-b7e6600057f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.387133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:00:56.400613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89aba0d-8101-4bcc-b04b-d753805fbbb9 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:56.404355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f5b981-dcc9-4ad1-87ce-f24743ab0df7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:56.405057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5b981-dcc9-4ad1-87ce-f24743ab0df7 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:56.405343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5b981-dcc9-4ad1-87ce-f24743ab0df7 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:56.405629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f5b981-dcc9-4ad1-87ce-f24743ab0df7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:56.414156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5b981-dcc9-4ad1-87ce-f24743ab0df7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:56.416327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f5b981-dcc9-4ad1-87ce-f24743ab0df7 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:56.425312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1029/2035] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:56.427818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4761-810d-48b4-bd14-b7e6600057f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:56.429356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d4761-810d-48b4-bd14-b7e6600057f7 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:56.439154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1007/2036] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:00:57.703260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca1ded-0bd0-48c9-a4af-a20e6875e405 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:57.703771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca1ded-0bd0-48c9-a4af-a20e6875e405 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:57.703957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca1ded-0bd0-48c9-a4af-a20e6875e405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:57.732445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9aca1ded-0bd0-48c9-a4af-a20e6875e405 None None] MFA Rules not processed for user `5caf24262d214d9aa8cf9612582154f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:57.733684 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9aca1ded-0bd0-48c9-a4af-a20e6875e405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:57.807181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1030/2037] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:00:57.817773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4761-810d-48b4-bd14-b7e6600057f7 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:57.826572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d4761-810d-48b4-bd14-b7e6600057f7 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:57.830274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51e236-33f5-401d-a814-84f0ac06073e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:57.830925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e236-33f5-401d-a814-84f0ac06073e service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:57.831212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e236-33f5-401d-a814-84f0ac06073e service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:57.831518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51e236-33f5-401d-a814-84f0ac06073e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:57.840041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51e236-33f5-401d-a814-84f0ac06073e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:57.841498 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51e236-33f5-401d-a814-84f0ac06073e service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:57.850442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1008/2038] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:00:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:00:59.853210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440f9f2-4576-48a3-97b4-dbefcc9d3571 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:59.853679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440f9f2-4576-48a3-97b4-dbefcc9d3571 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:59.853809 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8440f9f2-4576-48a3-97b4-dbefcc9d3571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:59.858238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51e236-33f5-401d-a814-84f0ac06073e service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:00:59.870209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51e236-33f5-401d-a814-84f0ac06073e service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:00:59.874175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-8440f9f2-4576-48a3-97b4-dbefcc9d3571 None None] MFA Rules not processed for user `5caf24262d214d9aa8cf9612582154f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:00:59.874912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:00:59.875368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8440f9f2-4576-48a3-97b4-dbefcc9d3571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:00:59.875777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:00:59.876059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:00:59.876370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] PATH_INFO: `/v3/users/fee17de0ec784ddda697e3ccc6cd1f65` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:00:59.877570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee17de0ec784ddda697e3ccc6cd1f65)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:00:59.879519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:00:59.882327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1031/2039] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:00:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:00.081042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee17de0ec784ddda697e3ccc6cd1f65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.087458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440f9f2-4576-48a3-97b4-dbefcc9d3571 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:00.088878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8440f9f2-4576-48a3-97b4-dbefcc9d3571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.102320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8440f9f2-4576-48a3-97b4-dbefcc9d3571 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:00.107788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:00.108932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.109217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.109500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] PATH_INFO: `/v3/users/2cb62ed0a9604c5fbd24196709e1e1f8` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.110712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb62ed0a9604c5fbd24196709e1e1f8)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:00.112697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:00.155956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee17de0ec784ddda697e3ccc6cd1f65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.157634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee17de0ec784ddda697e3ccc6cd1f65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.166323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee17de0ec784ddda697e3ccc6cd1f65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.167014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4894f8e2-ce1c-4e93-94ff-702b6fcefe9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee17de0ec784ddda697e3ccc6cd1f65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.186737 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1009/2040] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:00:59 2022] DELETE /identity/v3/users/fee17de0ec784ddda697e3ccc6cd1f65 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:00.190920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d151ff-9c19-4314-a788-68114f27bc23 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.191211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d151ff-9c19-4314-a788-68114f27bc23 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.191492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d151ff-9c19-4314-a788-68114f27bc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.209223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f6d151ff-9c19-4314-a788-68114f27bc23 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:00.210507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f6d151ff-9c19-4314-a788-68114f27bc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.216156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:00.272241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1010/2041] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:00.282033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d151ff-9c19-4314-a788-68114f27bc23 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:00.283479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f6d151ff-9c19-4314-a788-68114f27bc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.336633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb62ed0a9604c5fbd24196709e1e1f8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.346258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d151ff-9c19-4314-a788-68114f27bc23 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:00.350378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec9efa7-7a78-4716-aff4-8344112163f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:00.351066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9efa7-7a78-4716-aff4-8344112163f7 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.355130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9efa7-7a78-4716-aff4-8344112163f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.355130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9efa7-7a78-4716-aff4-8344112163f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.363617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9efa7-7a78-4716-aff4-8344112163f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:00.365324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec9efa7-7a78-4716-aff4-8344112163f7 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:00.377070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1011/2042] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:00.401378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb62ed0a9604c5fbd24196709e1e1f8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.402007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb62ed0a9604c5fbd24196709e1e1f8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.410013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb62ed0a9604c5fbd24196709e1e1f8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.410901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb62ed0a9604c5fbd24196709e1e1f8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:00.430391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1032/2043] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:00 2022] DELETE /identity/v3/users/2cb62ed0a9604c5fbd24196709e1e1f8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:00.435133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ad22-f193-4b26-9064-7d029cb91264 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.435432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ad22-f193-4b26-9064-7d029cb91264 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.435743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9ad22-f193-4b26-9064-7d029cb91264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.453810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3ff9ad22-f193-4b26-9064-7d029cb91264 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:00.454939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3ff9ad22-f193-4b26-9064-7d029cb91264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.515580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1012/2044] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:00.526221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:00.527710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:00.594395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:00.628308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de903ef4-ee3e-4a16-a30d-3fa4ea0bc711 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:00.634038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:00.637108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:00.637108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:00.637108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:00.652944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:00.655124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:00.669074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1033/2045] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:01:01.069477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ad22-f193-4b26-9064-7d029cb91264 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.073250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3ff9ad22-f193-4b26-9064-7d029cb91264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:01.087295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9ad22-f193-4b26-9064-7d029cb91264 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.093213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.094378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.094755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.095103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] PATH_INFO: `/v3/projects/50d4a8f19ba9426095868160e165016b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.096414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=50d4a8f19ba9426095868160e165016b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.098395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.214073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.215585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:01.230103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0054d8-346b-409b-98b3-ee3b20809aeb service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.235359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.242099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.242099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.242099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] PATH_INFO: `/v3/projects/c73436901fc24d3cac76e4110b7ae237` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.242099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c73436901fc24d3cac76e4110b7ae237)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.242099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.276835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50d4a8f19ba9426095868160e165016b', 'request_id': 'req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.278494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:01.348011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c73436901fc24d3cac76e4110b7ae237', 'request_id': 'req-69a0c314-6bd4-44e2-8908-37f6ea285b99'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.388042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50d4a8f19ba9426095868160e165016b', 'request_id': 'req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.394286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50d4a8f19ba9426095868160e165016b', 'request_id': 'req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.405380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] The token cache is being invalidate because project 50d4a8f19ba9426095868160e165016b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:01.405663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.406299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b469a3e2-bd85-4ecc-bed2-3569b05ab75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.407845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1013/2046] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:01 2022] DELETE /identity/v3/projects/50d4a8f19ba9426095868160e165016b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:01.412604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c548f-850b-4135-a788-3b9214285ab0 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.412891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c548f-850b-4135-a788-3b9214285ab0 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.413166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c548f-850b-4135-a788-3b9214285ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.433275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-925c548f-850b-4135-a788-3b9214285ab0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:01.434553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-925c548f-850b-4135-a788-3b9214285ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:01.493177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1014/2047] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:01.496547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c73436901fc24d3cac76e4110b7ae237', 'request_id': 'req-69a0c314-6bd4-44e2-8908-37f6ea285b99'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.501288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66390259-c646-42ac-8abc-bbbc7bee91c3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.501565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66390259-c646-42ac-8abc-bbbc7bee91c3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.501846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66390259-c646-42ac-8abc-bbbc7bee91c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.502984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c73436901fc24d3cac76e4110b7ae237', 'request_id': 'req-69a0c314-6bd4-44e2-8908-37f6ea285b99'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.515168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] The token cache is being invalidate because project c73436901fc24d3cac76e4110b7ae237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:01.515455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.516101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:01.517685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1034/2048] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:01 2022] DELETE /identity/v3/projects/c73436901fc24d3cac76e4110b7ae237 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:01.521013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-66390259-c646-42ac-8abc-bbbc7bee91c3 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:01.522249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-66390259-c646-42ac-8abc-bbbc7bee91c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:01.531756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1015/2049] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:01.536265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.548024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0c314-6bd4-44e2-8908-37f6ea285b99 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.553232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.553921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.554199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.554541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.555296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:01.555723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.558170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.583514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1035/2050] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:01.585308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66390259-c646-42ac-8abc-bbbc7bee91c3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.595836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66390259-c646-42ac-8abc-bbbc7bee91c3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.600863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d10623-a24f-4b61-ae66-79d269dfb0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.601567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d10623-a24f-4b61-ae66-79d269dfb0fe admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.601852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d10623-a24f-4b61-ae66-79d269dfb0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.602130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d10623-a24f-4b61-ae66-79d269dfb0fe admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.604410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d10623-a24f-4b61-ae66-79d269dfb0fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.606398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d10623-a24f-4b61-ae66-79d269dfb0fe admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.745902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1016/2051] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:01 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:01.752730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.772985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4ee73-6b50-4f34-ac18-3f996856c1b1 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.848768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a6d8c-289c-4995-adbf-93f688dc45cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.849475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a6d8c-289c-4995-adbf-93f688dc45cd admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.849764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a6d8c-289c-4995-adbf-93f688dc45cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.858649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a6d8c-289c-4995-adbf-93f688dc45cd admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.858649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a6d8c-289c-4995-adbf-93f688dc45cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.858649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a6d8c-289c-4995-adbf-93f688dc45cd admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.927004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1036/2052] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:01 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:01.933686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d10623-a24f-4b61-ae66-79d269dfb0fe admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.944847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d10623-a24f-4b61-ae66-79d269dfb0fe admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.950782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526d58f-cc78-4ce8-a670-f4dcbbdd1398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.951507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526d58f-cc78-4ce8-a670-f4dcbbdd1398 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.951796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526d58f-cc78-4ce8-a670-f4dcbbdd1398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.952078 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526d58f-cc78-4ce8-a670-f4dcbbdd1398 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.952861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526d58f-cc78-4ce8-a670-f4dcbbdd1398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.955922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526d58f-cc78-4ce8-a670-f4dcbbdd1398 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:01.972054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1017/2053] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:01.975597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a6d8c-289c-4995-adbf-93f688dc45cd admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:01.987125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a6d8c-289c-4995-adbf-93f688dc45cd admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:01.992159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d116de2-fe96-4e8a-bdae-96e67f3e5c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:01.992834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d116de2-fe96-4e8a-bdae-96e67f3e5c57 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:01.993172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d116de2-fe96-4e8a-bdae-96e67f3e5c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:01.993555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d116de2-fe96-4e8a-bdae-96e67f3e5c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:01.994410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d116de2-fe96-4e8a-bdae-96e67f3e5c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:01.996566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d116de2-fe96-4e8a-bdae-96e67f3e5c57 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:02.009795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1037/2054] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:02.013887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526d58f-cc78-4ce8-a670-f4dcbbdd1398 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:02.027032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526d58f-cc78-4ce8-a670-f4dcbbdd1398 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:02.032384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:02.033559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:02.033746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:02.034026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] PATH_INFO: `/v3/projects/49decbc30e5c4ba8bb7cd58d6b997974/users/d24564c93fff4923a1e1a5e0399f36a4/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:02.055075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49decbc30e5c4ba8bb7cd58d6b997974, user_id=d24564c93fff4923a1e1a5e0399f36a4, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:02.058451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:02.158166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1018/2055] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:02 2022] PUT /identity/v3/projects/49decbc30e5c4ba8bb7cd58d6b997974/users/d24564c93fff4923a1e1a5e0399f36a4/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:02.173638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec9b7-7d4e-4116-a47e-d437a5701e32 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:02.173930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec9b7-7d4e-4116-a47e-d437a5701e32 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:02.174205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec9b7-7d4e-4116-a47e-d437a5701e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:02.207614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4b3ec9b7-7d4e-4116-a47e-d437a5701e32 None None] MFA Rules not processed for user `d24564c93fff4923a1e1a5e0399f36a4`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:02.209897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4b3ec9b7-7d4e-4116-a47e-d437a5701e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:02.233989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:02.297795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1038/2056] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:02.309741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:02.312496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:02.388501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a56923-41e0-4662-bc87-dbc52749e011 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:02.394000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da2267-887b-46ac-83e9-1b1deb84cb77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:02.396137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da2267-887b-46ac-83e9-1b1deb84cb77 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:02.396542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da2267-887b-46ac-83e9-1b1deb84cb77 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:02.397872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da2267-887b-46ac-83e9-1b1deb84cb77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:02.410983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da2267-887b-46ac-83e9-1b1deb84cb77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:02.414188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da2267-887b-46ac-83e9-1b1deb84cb77 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:02.432378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1019/2057] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:07.044651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec9b7-7d4e-4116-a47e-d437a5701e32 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:07.046062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4b3ec9b7-7d4e-4116-a47e-d437a5701e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:07.110410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec9b7-7d4e-4116-a47e-d437a5701e32 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:07.115006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:07.115887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:07.116238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:07.124225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] PATH_INFO: `/v3/users/96ea34f608184b31bdb0f155bab10301` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:07.124225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ea34f608184b31bdb0f155bab10301)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:07.124225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:07.373903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ea34f608184b31bdb0f155bab10301'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:07.438976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ea34f608184b31bdb0f155bab10301'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:07.439597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ea34f608184b31bdb0f155bab10301'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:07.447524 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ea34f608184b31bdb0f155bab10301'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:07.450754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ea34f608184b31bdb0f155bab10301'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:07.466064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1039/2058] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:07 2022] DELETE /identity/v3/users/96ea34f608184b31bdb0f155bab10301 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:07.470534 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd067-a820-41ab-bcf9-dcf5814e3be4 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:07.470989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd067-a820-41ab-bcf9-dcf5814e3be4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:07.471102 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccd067-a820-41ab-bcf9-dcf5814e3be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:07.488801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b3ccd067-a820-41ab-bcf9-dcf5814e3be4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:07.490001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b3ccd067-a820-41ab-bcf9-dcf5814e3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:07.502212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:07.550050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1020/2059] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:07.560810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:07.562244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:07.605147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:07.621102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97654b23-726e-4b6e-8aa3-7e10e2edd285 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:07.624680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53d7fb-739c-469b-9ab9-552bd1d07330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:07.625333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53d7fb-739c-469b-9ab9-552bd1d07330 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:07.625623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53d7fb-739c-469b-9ab9-552bd1d07330 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:07.625908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53d7fb-739c-469b-9ab9-552bd1d07330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:07.636291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53d7fb-739c-469b-9ab9-552bd1d07330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:07.638147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53d7fb-739c-469b-9ab9-552bd1d07330 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:07.649063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1040/2060] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:08.077013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd067-a820-41ab-bcf9-dcf5814e3be4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.078444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b3ccd067-a820-41ab-bcf9-dcf5814e3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:08.092292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccd067-a820-41ab-bcf9-dcf5814e3be4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.097754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.098660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.098960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.099238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] PATH_INFO: `/v3/projects/fe6bc36a72b14577ad900d4ba4e1c382` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.100468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6bc36a72b14577ad900d4ba4e1c382)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:08.103842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:08.287069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6bc36a72b14577ad900d4ba4e1c382', 'request_id': 'req-b4d0e21c-0daa-4f39-ba79-002e78947202'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.411489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6bc36a72b14577ad900d4ba4e1c382', 'request_id': 'req-b4d0e21c-0daa-4f39-ba79-002e78947202'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.417738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6bc36a72b14577ad900d4ba4e1c382', 'request_id': 'req-b4d0e21c-0daa-4f39-ba79-002e78947202'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.429320 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] The token cache is being invalidate because project fe6bc36a72b14577ad900d4ba4e1c382 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:08.429609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.430304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b4d0e21c-0daa-4f39-ba79-002e78947202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:08.431915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1021/2061] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:08 2022] DELETE /identity/v3/projects/fe6bc36a72b14577ad900d4ba4e1c382 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:08.436936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1bd76-bb67-4f37-87b2-8c271523f561 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.437440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1bd76-bb67-4f37-87b2-8c271523f561 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.437518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1bd76-bb67-4f37-87b2-8c271523f561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.456806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c1f1bd76-bb67-4f37-87b2-8c271523f561 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:08.458122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c1f1bd76-bb67-4f37-87b2-8c271523f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:08.519072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1041/2062] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:08.527654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb001d-c148-4f0c-8aef-0ee2fc608718 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.528199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb001d-c148-4f0c-8aef-0ee2fc608718 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.528296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb001d-c148-4f0c-8aef-0ee2fc608718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.547493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f8eb001d-c148-4f0c-8aef-0ee2fc608718 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:08.548867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f8eb001d-c148-4f0c-8aef-0ee2fc608718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:08.560307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1022/2063] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:08.565172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1bd76-bb67-4f37-87b2-8c271523f561 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.576706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1bd76-bb67-4f37-87b2-8c271523f561 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.581540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.582205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.582518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.582793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.583527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:08.583833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:08.585469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:08.598272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1042/2064] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:08.606321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb001d-c148-4f0c-8aef-0ee2fc608718 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.616524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb001d-c148-4f0c-8aef-0ee2fc608718 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.622225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6df59-f76e-4a2e-a3f6-80e217897217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.622960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6df59-f76e-4a2e-a3f6-80e217897217 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.623355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6df59-f76e-4a2e-a3f6-80e217897217 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.623638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6df59-f76e-4a2e-a3f6-80e217897217 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.624369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6df59-f76e-4a2e-a3f6-80e217897217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:08.626314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6df59-f76e-4a2e-a3f6-80e217897217 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:08.727237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1023/2065] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:08 2022] POST /identity/v3/projects => generated 405 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:08.731860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.741876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62ed4a-27b5-4992-ab34-5e9c0aa1fd98 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.802008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b9072-1b56-4150-90bf-072badc908fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.802691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b9072-1b56-4150-90bf-072badc908fc admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.802980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b9072-1b56-4150-90bf-072badc908fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.803268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b9072-1b56-4150-90bf-072badc908fc admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.803983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b9072-1b56-4150-90bf-072badc908fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:08.806661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419b9072-1b56-4150-90bf-072badc908fc admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:08.894381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1043/2066] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:08 2022] POST /identity/v3/users => generated 303 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:08.895702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6df59-f76e-4a2e-a3f6-80e217897217 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.906865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6df59-f76e-4a2e-a3f6-80e217897217 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.911642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f7612-736b-436a-a12d-a87401440669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.912365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f7612-736b-436a-a12d-a87401440669 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.912653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f7612-736b-436a-a12d-a87401440669 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.912941 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f7612-736b-436a-a12d-a87401440669 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.913712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f7612-736b-436a-a12d-a87401440669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:08.915996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f7612-736b-436a-a12d-a87401440669 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:08.926735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1024/2067] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:08.931604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b9072-1b56-4150-90bf-072badc908fc admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.942876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419b9072-1b56-4150-90bf-072badc908fc admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.947961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da4acc-dadf-4188-8783-2cdcacbccdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.948664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da4acc-dadf-4188-8783-2cdcacbccdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.948955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da4acc-dadf-4188-8783-2cdcacbccdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.949235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da4acc-dadf-4188-8783-2cdcacbccdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:08.949966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da4acc-dadf-4188-8783-2cdcacbccdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:08.952579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da4acc-dadf-4188-8783-2cdcacbccdf6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:08.963403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1044/2068] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:08.967661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f7612-736b-436a-a12d-a87401440669 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:08.981269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f7612-736b-436a-a12d-a87401440669 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:08.985886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:08.986775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:08.987058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:08.987334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] PATH_INFO: `/v3/projects/ff0ba8da3c914104b7e591b9d854878d/users/a357931f82554abf891d7494b783a28b/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:09.002316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0ba8da3c914104b7e591b9d854878d, user_id=a357931f82554abf891d7494b783a28b, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:09.005000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:09.090997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1025/2069] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:08 2022] PUT /identity/v3/projects/ff0ba8da3c914104b7e591b9d854878d/users/a357931f82554abf891d7494b783a28b/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:09.096101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae29d4-be19-4a6c-a4d9-1cdc621e4e0d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:09.096424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae29d4-be19-4a6c-a4d9-1cdc621e4e0d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:09.096707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae29d4-be19-4a6c-a4d9-1cdc621e4e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:09.115216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-5aae29d4-be19-4a6c-a4d9-1cdc621e4e0d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:09.116461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5aae29d4-be19-4a6c-a4d9-1cdc621e4e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:09.158373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:09.175978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1045/2070] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:09.186351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:09.188204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:09.232802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:09.247273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4538cb-5da2-41a8-9de7-77959f59c934 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:09.250990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3339ea-068a-45a5-97c2-11e1d414dacf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:09.251636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3339ea-068a-45a5-97c2-11e1d414dacf service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:09.256895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3339ea-068a-45a5-97c2-11e1d414dacf service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:09.256895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3339ea-068a-45a5-97c2-11e1d414dacf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:09.261905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3339ea-068a-45a5-97c2-11e1d414dacf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:09.263329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3339ea-068a-45a5-97c2-11e1d414dacf service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:09.272853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1026/2071] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:10.764013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9a0c1-27cd-4b03-9846-c8fb60d3a424 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:10.764518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9a0c1-27cd-4b03-9846-c8fb60d3a424 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:10.764651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9a0c1-27cd-4b03-9846-c8fb60d3a424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:10.793468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-95a9a0c1-27cd-4b03-9846-c8fb60d3a424 None None] MFA Rules not processed for user `a357931f82554abf891d7494b783a28b`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:10.794713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-95a9a0c1-27cd-4b03-9846-c8fb60d3a424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:10.910843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1046/2072] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:10.927905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3339ea-068a-45a5-97c2-11e1d414dacf service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:10.938900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3339ea-068a-45a5-97c2-11e1d414dacf service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:10.943683 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e8ddc-04ce-4f0c-b7f0-382abb0c3b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:10.944452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e8ddc-04ce-4f0c-b7f0-382abb0c3b2b admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:10.944806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e8ddc-04ce-4f0c-b7f0-382abb0c3b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:10.945148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e8ddc-04ce-4f0c-b7f0-382abb0c3b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:10.946785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e8ddc-04ce-4f0c-b7f0-382abb0c3b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:10.949126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e8ddc-04ce-4f0c-b7f0-382abb0c3b2b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:11.195755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1027/2073] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:10 2022] POST /identity/v3/projects => generated 405 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:11.198933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9a0c1-27cd-4b03-9846-c8fb60d3a424 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:11.210294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9a0c1-27cd-4b03-9846-c8fb60d3a424 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:11.275997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a58ba7-e6a6-4adc-beda-08660d80a8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:11.276864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a58ba7-e6a6-4adc-beda-08660d80a8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:11.277151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a58ba7-e6a6-4adc-beda-08660d80a8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:11.277491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a58ba7-e6a6-4adc-beda-08660d80a8f8 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:11.278193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a58ba7-e6a6-4adc-beda-08660d80a8f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:11.281318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a58ba7-e6a6-4adc-beda-08660d80a8f8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:11.411828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1047/2074] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:11 2022] POST /identity/v3/users => generated 309 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:11.416693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e8ddc-04ce-4f0c-b7f0-382abb0c3b2b admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:11.427339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e8ddc-04ce-4f0c-b7f0-382abb0c3b2b admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:11.432989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a8188-4674-4202-826d-b9253cd0fa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:11.433668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a8188-4674-4202-826d-b9253cd0fa4d admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:11.433972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a8188-4674-4202-826d-b9253cd0fa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:11.434258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a8188-4674-4202-826d-b9253cd0fa4d admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:11.435003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a8188-4674-4202-826d-b9253cd0fa4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:11.436658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a8188-4674-4202-826d-b9253cd0fa4d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:11.447790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1028/2075] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:11.452816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a58ba7-e6a6-4adc-beda-08660d80a8f8 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:11.466439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a58ba7-e6a6-4adc-beda-08660d80a8f8 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:11.470440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f65bdc-0f25-4941-82dc-c6f97d6edcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:11.471333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f65bdc-0f25-4941-82dc-c6f97d6edcb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:11.471626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f65bdc-0f25-4941-82dc-c6f97d6edcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:11.471901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f65bdc-0f25-4941-82dc-c6f97d6edcb2 admin admin] PATH_INFO: `/v3/projects/7a8803ada7a846c289921222d0bed5ce/users/680db6c8ff044f92ba7f3462b68a1455/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:11.488763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f65bdc-0f25-4941-82dc-c6f97d6edcb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8803ada7a846c289921222d0bed5ce, user_id=680db6c8ff044f92ba7f3462b68a1455, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:11.491565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f65bdc-0f25-4941-82dc-c6f97d6edcb2 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:11.579952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1048/2076] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:11 2022] PUT /identity/v3/projects/7a8803ada7a846c289921222d0bed5ce/users/680db6c8ff044f92ba7f3462b68a1455/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:11.584067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a8188-4674-4202-826d-b9253cd0fa4d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:11.596520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a8188-4674-4202-826d-b9253cd0fa4d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:11.661051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf3076-3c77-4a46-96f9-ef53b27893a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:11.661733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf3076-3c77-4a46-96f9-ef53b27893a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:11.662017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf3076-3c77-4a46-96f9-ef53b27893a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:11.662299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf3076-3c77-4a46-96f9-ef53b27893a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:11.663036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf3076-3c77-4a46-96f9-ef53b27893a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:11.665510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf3076-3c77-4a46-96f9-ef53b27893a6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:11.676251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1029/2077] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:11.680822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f65bdc-0f25-4941-82dc-c6f97d6edcb2 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:11.692046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f65bdc-0f25-4941-82dc-c6f97d6edcb2 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:11.697228 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0437503-ab18-47b9-95ec-dbf3f411dee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:11.698137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0437503-ab18-47b9-95ec-dbf3f411dee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:11.698424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0437503-ab18-47b9-95ec-dbf3f411dee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:11.698711 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0437503-ab18-47b9-95ec-dbf3f411dee0 admin admin] PATH_INFO: `/v3/projects/7a8803ada7a846c289921222d0bed5ce/users/680db6c8ff044f92ba7f3462b68a1455/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:11.700824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0437503-ab18-47b9-95ec-dbf3f411dee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a8803ada7a846c289921222d0bed5ce, user_id=680db6c8ff044f92ba7f3462b68a1455, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:11.703543 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0437503-ab18-47b9-95ec-dbf3f411dee0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:11.715975 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1049/2078] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:11 2022] PUT /identity/v3/projects/7a8803ada7a846c289921222d0bed5ce/users/680db6c8ff044f92ba7f3462b68a1455/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:12.944636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92779c-98e6-4799-be5c-041b4c56f633 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:12.945149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92779c-98e6-4799-be5c-041b4c56f633 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:12.945236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92779c-98e6-4799-be5c-041b4c56f633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:12.974464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-bf92779c-98e6-4799-be5c-041b4c56f633 None None] MFA Rules not processed for user `680db6c8ff044f92ba7f3462b68a1455`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:12.975708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bf92779c-98e6-4799-be5c-041b4c56f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:13.052704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1030/2079] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:12 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:14.370506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd5f88-175a-40a0-98b1-3f91194d8ff8 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:14.371033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd5f88-175a-40a0-98b1-3f91194d8ff8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:14.371356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbd5f88-175a-40a0-98b1-3f91194d8ff8 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:14.373066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1050/2080] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 06:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:01:14.404578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb89c7-ffc9-4742-9563-75f8ce1bb03b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:14.405200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb89c7-ffc9-4742-9563-75f8ce1bb03b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:14.405200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeb89c7-ffc9-4742-9563-75f8ce1bb03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:14.425463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2eeb89c7-ffc9-4742-9563-75f8ce1bb03b None None] MFA Rules not processed for user `7c4988dc5513476786f11d08059c3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:14.426668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2eeb89c7-ffc9-4742-9563-75f8ce1bb03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:14.452143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:14.499369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1031/2081] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 06:01:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:14.513707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd5f88-175a-40a0-98b1-3f91194d8ff8 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:14.515651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9fbd5f88-175a-40a0-98b1-3f91194d8ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:14.534588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb89c7-ffc9-4742-9563-75f8ce1bb03b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:14.540851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbd5f88-175a-40a0-98b1-3f91194d8ff8 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:14.558183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d4b5e-f4e3-49c6-9286-353bd95f6fcf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:14.558900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d4b5e-f4e3-49c6-9286-353bd95f6fcf service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:14.559178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d4b5e-f4e3-49c6-9286-353bd95f6fcf service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:14.559480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d4b5e-f4e3-49c6-9286-353bd95f6fcf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:14.568394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d4b5e-f4e3-49c6-9286-353bd95f6fcf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:14.570013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d4b5e-f4e3-49c6-9286-353bd95f6fcf service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:14.580067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1051/2082] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:14.603395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeb89c7-ffc9-4742-9563-75f8ce1bb03b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:14.608913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:14.609834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:14.610122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:14.610400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] PATH_INFO: `/v3/users/a357931f82554abf891d7494b783a28b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:14.611657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a357931f82554abf891d7494b783a28b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:14.614184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:14.797001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a357931f82554abf891d7494b783a28b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:14.885858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a357931f82554abf891d7494b783a28b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:14.886524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a357931f82554abf891d7494b783a28b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:14.894272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a357931f82554abf891d7494b783a28b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:14.896725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a357931f82554abf891d7494b783a28b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:14.913505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1032/2083] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:14 2022] DELETE /identity/v3/users/a357931f82554abf891d7494b783a28b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:15.167942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d4b5e-f4e3-49c6-9286-353bd95f6fcf service placement] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:15.178477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d4b5e-f4e3-49c6-9286-353bd95f6fcf service placement] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:15.191332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:15.234994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:15.235849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:15.236153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:15.236432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] PATH_INFO: `/v3/projects/ff0ba8da3c914104b7e591b9d854878d` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:15.237601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0ba8da3c914104b7e591b9d854878d)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:15.239496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:15.355305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0ba8da3c914104b7e591b9d854878d', 'request_id': 'req-8391438f-0eb0-411f-9dbe-01ac546c63db'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.405505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0ba8da3c914104b7e591b9d854878d', 'request_id': 'req-8391438f-0eb0-411f-9dbe-01ac546c63db'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.411370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0ba8da3c914104b7e591b9d854878d', 'request_id': 'req-8391438f-0eb0-411f-9dbe-01ac546c63db'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.423947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] The token cache is being invalidate because project ff0ba8da3c914104b7e591b9d854878d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:15.424262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.424888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.426410 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1052/2084] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:15 2022] DELETE /identity/v3/projects/ff0ba8da3c914104b7e591b9d854878d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:15.430721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:15.434063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:15.444779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f568cd0-ff14-496a-aba4-5bea49388d65 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:15.486051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:15.500515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:15.501404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:15.501690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:15.501969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] PATH_INFO: `/v3/users/680db6c8ff044f92ba7f3462b68a1455` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:15.503167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680db6c8ff044f92ba7f3462b68a1455)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:15.505140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:15.714597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680db6c8ff044f92ba7f3462b68a1455'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.794079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680db6c8ff044f92ba7f3462b68a1455'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.794828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680db6c8ff044f92ba7f3462b68a1455'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.810139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680db6c8ff044f92ba7f3462b68a1455'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.811180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c18e4fbc-029f-4674-9bc4-78a640719fc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680db6c8ff044f92ba7f3462b68a1455'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:15.829440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1033/2085] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:15 2022] DELETE /identity/v3/users/680db6c8ff044f92ba7f3462b68a1455 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:16.071456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:16.074422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:16.086438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391438f-0eb0-411f-9dbe-01ac546c63db admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:16.168863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:16.169624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.169909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.170190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] PATH_INFO: `/v3/projects/7a8803ada7a846c289921222d0bed5ce` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.171386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a8803ada7a846c289921222d0bed5ce)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:16.173343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:16.279085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:16.280359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a8803ada7a846c289921222d0bed5ce', 'request_id': 'req-1d13501a-90a6-4e25-953e-52a0c8b39ba2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:16.355157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a8803ada7a846c289921222d0bed5ce', 'request_id': 'req-1d13501a-90a6-4e25-953e-52a0c8b39ba2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:16.361110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a8803ada7a846c289921222d0bed5ce', 'request_id': 'req-1d13501a-90a6-4e25-953e-52a0c8b39ba2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:16.372698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] The token cache is being invalidate because project 7a8803ada7a846c289921222d0bed5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:16.372987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:16.373627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d13501a-90a6-4e25-953e-52a0c8b39ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:16.375173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1053/2086] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:16 2022] DELETE /identity/v3/projects/7a8803ada7a846c289921222d0bed5ce => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:16.380048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7653b-d2f5-47f2-8d0c-9928470d4ca8 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.380432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7653b-d2f5-47f2-8d0c-9928470d4ca8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.380779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7653b-d2f5-47f2-8d0c-9928470d4ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.400300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-39a7653b-d2f5-47f2-8d0c-9928470d4ca8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:16.401510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-39a7653b-d2f5-47f2-8d0c-9928470d4ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:16.462406 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1034/2087] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:16.469972 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a57d-c409-43cd-80dc-6978ca8df172 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.470371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a57d-c409-43cd-80dc-6978ca8df172 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.470536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a57d-c409-43cd-80dc-6978ca8df172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.489263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fe67a57d-c409-43cd-80dc-6978ca8df172 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:16.490503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fe67a57d-c409-43cd-80dc-6978ca8df172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:16.499359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1054/2088] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:16.503658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7653b-d2f5-47f2-8d0c-9928470d4ca8 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:16.514516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7653b-d2f5-47f2-8d0c-9928470d4ca8 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:16.519309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:16.519981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.520314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.520643 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.521440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:16.522010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:16.523728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:16.540409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1035/2089] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:16.541608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67a57d-c409-43cd-80dc-6978ca8df172 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:16.551247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67a57d-c409-43cd-80dc-6978ca8df172 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:16.556964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c3dc-ec10-4e99-b9fe-7e8e91ccf245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:16.557701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c3dc-ec10-4e99-b9fe-7e8e91ccf245 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.558058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c3dc-ec10-4e99-b9fe-7e8e91ccf245 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.558411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c3dc-ec10-4e99-b9fe-7e8e91ccf245 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.559121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c3dc-ec10-4e99-b9fe-7e8e91ccf245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:16.561071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c3dc-ec10-4e99-b9fe-7e8e91ccf245 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:16.679353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1055/2090] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:16 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:16.681381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:16.691638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6b692-729c-460e-b172-69ea2e99f553 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:16.738014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:16.748404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bdf2e-327f-40f4-a5a8-5e56ce6b9006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:16.749292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bdf2e-327f-40f4-a5a8-5e56ce6b9006 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.749574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bdf2e-327f-40f4-a5a8-5e56ce6b9006 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.749848 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bdf2e-327f-40f4-a5a8-5e56ce6b9006 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.750541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bdf2e-327f-40f4-a5a8-5e56ce6b9006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:16.752540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bdf2e-327f-40f4-a5a8-5e56ce6b9006 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:16.861599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1036/2091] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:16 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:16.862922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c3dc-ec10-4e99-b9fe-7e8e91ccf245 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:16.878335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c3dc-ec10-4e99-b9fe-7e8e91ccf245 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:16.878759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0cb60-7b40-4810-9e7f-9e1bdd15601c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:16.879436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0cb60-7b40-4810-9e7f-9e1bdd15601c admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.879721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0cb60-7b40-4810-9e7f-9e1bdd15601c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.880020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0cb60-7b40-4810-9e7f-9e1bdd15601c admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.880925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0cb60-7b40-4810-9e7f-9e1bdd15601c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:16.882537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf0cb60-7b40-4810-9e7f-9e1bdd15601c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:16.893863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1056/2092] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:16.898860 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bdf2e-327f-40f4-a5a8-5e56ce6b9006 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:16.909114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bdf2e-327f-40f4-a5a8-5e56ce6b9006 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:16.913830 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8b8e6-2ef3-4071-8424-6cf2a00ca6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:16.915158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8b8e6-2ef3-4071-8424-6cf2a00ca6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.915445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8b8e6-2ef3-4071-8424-6cf2a00ca6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.915727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8b8e6-2ef3-4071-8424-6cf2a00ca6f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.916528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8b8e6-2ef3-4071-8424-6cf2a00ca6f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:16.918125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8b8e6-2ef3-4071-8424-6cf2a00ca6f5 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:16.928921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1037/2093] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:16.933203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0cb60-7b40-4810-9e7f-9e1bdd15601c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:16.943298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0cb60-7b40-4810-9e7f-9e1bdd15601c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:16.948322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aa0f1-38c6-4b77-86fe-271839fdd914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:16.949204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aa0f1-38c6-4b77-86fe-271839fdd914 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:16.949494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aa0f1-38c6-4b77-86fe-271839fdd914 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:16.950356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831aa0f1-38c6-4b77-86fe-271839fdd914 admin admin] PATH_INFO: `/v3/projects/d5ed0b7603db40ab8fe7e2c7d8ef47f1/users/aaedb2d0fbc949edb1cfa614eac6701e/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:16.965515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831aa0f1-38c6-4b77-86fe-271839fdd914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ed0b7603db40ab8fe7e2c7d8ef47f1, user_id=aaedb2d0fbc949edb1cfa614eac6701e, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:16.968238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831aa0f1-38c6-4b77-86fe-271839fdd914 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:17.063261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1057/2094] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:16 2022] PUT /identity/v3/projects/d5ed0b7603db40ab8fe7e2c7d8ef47f1/users/aaedb2d0fbc949edb1cfa614eac6701e/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:17.075048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06415117-5ac0-4558-84de-77bcd701b51a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:17.075340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06415117-5ac0-4558-84de-77bcd701b51a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:17.075615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06415117-5ac0-4558-84de-77bcd701b51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:17.102824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-06415117-5ac0-4558-84de-77bcd701b51a None None] MFA Rules not processed for user `aaedb2d0fbc949edb1cfa614eac6701e`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:17.104056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-06415117-5ac0-4558-84de-77bcd701b51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:17.172925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1038/2095] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:17.177280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aa0f1-38c6-4b77-86fe-271839fdd914 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:17.244290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831aa0f1-38c6-4b77-86fe-271839fdd914 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:17.249048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2279c-9c9c-4103-87d7-b23fa0fe7f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:17.249769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2279c-9c9c-4103-87d7-b23fa0fe7f7c admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:17.250065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2279c-9c9c-4103-87d7-b23fa0fe7f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:17.250349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2279c-9c9c-4103-87d7-b23fa0fe7f7c admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:17.251051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2279c-9c9c-4103-87d7-b23fa0fe7f7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:17.253011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2279c-9c9c-4103-87d7-b23fa0fe7f7c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:17.354296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1058/2096] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:17 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:17.356946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06415117-5ac0-4558-84de-77bcd701b51a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:17.368206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06415117-5ac0-4558-84de-77bcd701b51a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:17.426780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bb485-9be3-49ef-bf23-321fa0bd6aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:17.427557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bb485-9be3-49ef-bf23-321fa0bd6aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:17.427910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bb485-9be3-49ef-bf23-321fa0bd6aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:17.428303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bb485-9be3-49ef-bf23-321fa0bd6aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:17.429073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bb485-9be3-49ef-bf23-321fa0bd6aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:17.431702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bb485-9be3-49ef-bf23-321fa0bd6aa5 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:17.527028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1039/2097] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:17 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:17.530975 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2279c-9c9c-4103-87d7-b23fa0fe7f7c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:17.542915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2279c-9c9c-4103-87d7-b23fa0fe7f7c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:17.547661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb480c-688d-4168-adbb-2743f480a22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:17.548362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb480c-688d-4168-adbb-2743f480a22b admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:17.548647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb480c-688d-4168-adbb-2743f480a22b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:17.548931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb480c-688d-4168-adbb-2743f480a22b admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:17.549664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb480c-688d-4168-adbb-2743f480a22b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:17.551260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb480c-688d-4168-adbb-2743f480a22b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:17.562968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1059/2098] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:17.567035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bb485-9be3-49ef-bf23-321fa0bd6aa5 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:17.577088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bb485-9be3-49ef-bf23-321fa0bd6aa5 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:17.582361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124b0c5-1d25-49df-aac6-7e49ff379274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:17.583356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b0c5-1d25-49df-aac6-7e49ff379274 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:17.583642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b0c5-1d25-49df-aac6-7e49ff379274 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:17.583967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124b0c5-1d25-49df-aac6-7e49ff379274 admin admin] PATH_INFO: `/v3/projects/be272f563af54995aec2657206de7f0f/users/94b9c347561847aea62617ef446a8d20/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:17.599592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124b0c5-1d25-49df-aac6-7e49ff379274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be272f563af54995aec2657206de7f0f, user_id=94b9c347561847aea62617ef446a8d20, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:17.603033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124b0c5-1d25-49df-aac6-7e49ff379274 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:17.720720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1040/2099] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:17 2022] PUT /identity/v3/projects/be272f563af54995aec2657206de7f0f/users/94b9c347561847aea62617ef446a8d20/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:17.724732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb480c-688d-4168-adbb-2743f480a22b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:17.736851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb480c-688d-4168-adbb-2743f480a22b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:17.786991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:17.800191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa76c7-4b30-4009-a08d-eb635758db84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:17.800872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa76c7-4b30-4009-a08d-eb635758db84 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:17.801159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa76c7-4b30-4009-a08d-eb635758db84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:17.801437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa76c7-4b30-4009-a08d-eb635758db84 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:17.802178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa76c7-4b30-4009-a08d-eb635758db84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:17.803898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa76c7-4b30-4009-a08d-eb635758db84 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:17.815312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1060/2100] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:17.819705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124b0c5-1d25-49df-aac6-7e49ff379274 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:17.831389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124b0c5-1d25-49df-aac6-7e49ff379274 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:17.836562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:17.838226 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:17.838226 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:17.838226 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] PATH_INFO: `/v3/projects/be272f563af54995aec2657206de7f0f/users/94b9c347561847aea62617ef446a8d20/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:17.840157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be272f563af54995aec2657206de7f0f, user_id=94b9c347561847aea62617ef446a8d20, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:17.842792 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:17.854127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1041/2101] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:17 2022] PUT /identity/v3/projects/be272f563af54995aec2657206de7f0f/users/94b9c347561847aea62617ef446a8d20/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:17.864279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ef3df-4e3a-49f6-98e9-ae481c653be0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:17.864574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ef3df-4e3a-49f6-98e9-ae481c653be0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:17.864851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ef3df-4e3a-49f6-98e9-ae481c653be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:17.891443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-cc7ef3df-4e3a-49f6-98e9-ae481c653be0 None None] MFA Rules not processed for user `94b9c347561847aea62617ef446a8d20`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:17.892675 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cc7ef3df-4e3a-49f6-98e9-ae481c653be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:17.969491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1061/2102] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:17 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:17.998195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:18.000485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:18.028332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:18.084681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1de66-3d09-4266-9aa7-345976d8e779 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:18.089789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07f53b-b19e-4471-ac9e-a196ac2d3c2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:18.090557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07f53b-b19e-4471-ac9e-a196ac2d3c2e service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:18.090841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07f53b-b19e-4471-ac9e-a196ac2d3c2e service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:18.091179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f07f53b-b19e-4471-ac9e-a196ac2d3c2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:18.103067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07f53b-b19e-4471-ac9e-a196ac2d3c2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:18.103067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f07f53b-b19e-4471-ac9e-a196ac2d3c2e service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:18.112023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1042/2103] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:17 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:18.214205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ef3df-4e3a-49f6-98e9-ae481c653be0 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:18.215724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cc7ef3df-4e3a-49f6-98e9-ae481c653be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:18.227936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ef3df-4e3a-49f6-98e9-ae481c653be0 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:18.231682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a74e7-f0e0-4407-a5c6-ad49ff8c6304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:18.232451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a74e7-f0e0-4407-a5c6-ad49ff8c6304 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:18.232744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a74e7-f0e0-4407-a5c6-ad49ff8c6304 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:18.233024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a74e7-f0e0-4407-a5c6-ad49ff8c6304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:18.293086 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a74e7-f0e0-4407-a5c6-ad49ff8c6304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:18.294562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a74e7-f0e0-4407-a5c6-ad49ff8c6304 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:18.303690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1062/2104] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:23.386666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07f53b-b19e-4471-ac9e-a196ac2d3c2e service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.452133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f07f53b-b19e-4471-ac9e-a196ac2d3c2e service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.456917 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.457789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.458086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.458342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] PATH_INFO: `/v3/users/aaedb2d0fbc949edb1cfa614eac6701e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.459558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaedb2d0fbc949edb1cfa614eac6701e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.461499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.648936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaedb2d0fbc949edb1cfa614eac6701e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:23.723827 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaedb2d0fbc949edb1cfa614eac6701e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:23.724418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaedb2d0fbc949edb1cfa614eac6701e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:23.731863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaedb2d0fbc949edb1cfa614eac6701e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:23.732529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaedb2d0fbc949edb1cfa614eac6701e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:23.750914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1043/2105] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:23 2022] DELETE /identity/v3/users/aaedb2d0fbc949edb1cfa614eac6701e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:23.755495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a66451-7172-44a6-a9af-94bdc82e2104 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.755984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a66451-7172-44a6-a9af-94bdc82e2104 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.756083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a66451-7172-44a6-a9af-94bdc82e2104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.779232 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-a3a66451-7172-44a6-a9af-94bdc82e2104 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:23.780494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a3a66451-7172-44a6-a9af-94bdc82e2104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:23.840517 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1063/2106] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:23.850260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:23.851681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:23.887303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:23.908386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5370b9e-58d9-4934-8502-54a3c9113df9 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:23.912080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c22d7-47ad-4eec-8e84-e89f55ebaf1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:23.912753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c22d7-47ad-4eec-8e84-e89f55ebaf1a service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:23.913038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c22d7-47ad-4eec-8e84-e89f55ebaf1a service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:23.913315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c22d7-47ad-4eec-8e84-e89f55ebaf1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:23.925875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c22d7-47ad-4eec-8e84-e89f55ebaf1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:23.927314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c22d7-47ad-4eec-8e84-e89f55ebaf1a service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:23.938369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1044/2107] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:24.331764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a66451-7172-44a6-a9af-94bdc82e2104 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:24.333335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a3a66451-7172-44a6-a9af-94bdc82e2104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:24.347632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a66451-7172-44a6-a9af-94bdc82e2104 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:24.352859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:24.353859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:24.358497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:24.358497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] PATH_INFO: `/v3/projects/d5ed0b7603db40ab8fe7e2c7d8ef47f1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:24.358497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ed0b7603db40ab8fe7e2c7d8ef47f1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:24.359534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:24.361075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:24.532386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ed0b7603db40ab8fe7e2c7d8ef47f1', 'request_id': 'req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.599257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ed0b7603db40ab8fe7e2c7d8ef47f1', 'request_id': 'req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.615121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ed0b7603db40ab8fe7e2c7d8ef47f1', 'request_id': 'req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.618821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] The token cache is being invalidate because project d5ed0b7603db40ab8fe7e2c7d8ef47f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:24.619109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.619785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.621638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1064/2108] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:24 2022] DELETE /identity/v3/projects/d5ed0b7603db40ab8fe7e2c7d8ef47f1 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:24.625587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c22d7-47ad-4eec-8e84-e89f55ebaf1a service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:24.636543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c22d7-47ad-4eec-8e84-e89f55ebaf1a service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:24.698495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:24.699768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:24.700051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:24.700394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] PATH_INFO: `/v3/users/94b9c347561847aea62617ef446a8d20` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:24.701618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b9c347561847aea62617ef446a8d20)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:24.704233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:24.865600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b9c347561847aea62617ef446a8d20'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.953574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b9c347561847aea62617ef446a8d20'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.954221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b9c347561847aea62617ef446a8d20'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.961691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b9c347561847aea62617ef446a8d20'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.965592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff324f99-5914-41eb-8d2a-0874d4e943f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b9c347561847aea62617ef446a8d20'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:24.980582 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1045/2109] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:24 2022] DELETE /identity/v3/users/94b9c347561847aea62617ef446a8d20 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:25.382140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:25.384167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:25.417243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98e40f-6c2a-453e-bfcc-e25f5aecc9df admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:25.484570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:25.485891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:25.486380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:25.486920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] PATH_INFO: `/v3/projects/be272f563af54995aec2657206de7f0f` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:25.488397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=be272f563af54995aec2657206de7f0f)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:25.490535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:25.593772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be272f563af54995aec2657206de7f0f', 'request_id': 'req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:25.684818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be272f563af54995aec2657206de7f0f', 'request_id': 'req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:25.691706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be272f563af54995aec2657206de7f0f', 'request_id': 'req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:25.705733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] The token cache is being invalidate because project be272f563af54995aec2657206de7f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:25.706037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:25.706670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-27d11354-ba1e-4d51-9b0e-98bcf04b0f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:25.708242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1065/2110] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:25 2022] DELETE /identity/v3/projects/be272f563af54995aec2657206de7f0f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:25.713127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbacac-8bd5-407a-a5a3-7691bae7ef46 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:25.714327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbacac-8bd5-407a-a5a3-7691bae7ef46 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:25.714327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbacac-8bd5-407a-a5a3-7691bae7ef46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:25.732795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-0ddbacac-8bd5-407a-a5a3-7691bae7ef46 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:25.734002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0ddbacac-8bd5-407a-a5a3-7691bae7ef46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:25.789585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:25.801151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1046/2111] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:25.810603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2721a2-41ff-4e7e-ade0-8721bb3a2034 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:25.810893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2721a2-41ff-4e7e-ade0-8721bb3a2034 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:25.811199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2721a2-41ff-4e7e-ade0-8721bb3a2034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:25.830756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-bc2721a2-41ff-4e7e-ade0-8721bb3a2034 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:25.831949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bc2721a2-41ff-4e7e-ade0-8721bb3a2034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:25.840200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1066/2112] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:25.844585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbacac-8bd5-407a-a5a3-7691bae7ef46 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:25.855067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbacac-8bd5-407a-a5a3-7691bae7ef46 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:25.859787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:25.860486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:25.860774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:25.861055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:25.861819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:25.862131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:25.863756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:25.876358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1047/2113] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:25.881264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2721a2-41ff-4e7e-ade0-8721bb3a2034 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:25.891242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2721a2-41ff-4e7e-ade0-8721bb3a2034 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:25.896011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05eaa1-dd86-427c-84a7-ed51d610e641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:25.897293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05eaa1-dd86-427c-84a7-ed51d610e641 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:25.897592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05eaa1-dd86-427c-84a7-ed51d610e641 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:25.897881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05eaa1-dd86-427c-84a7-ed51d610e641 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:25.898582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05eaa1-dd86-427c-84a7-ed51d610e641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:25.900617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05eaa1-dd86-427c-84a7-ed51d610e641 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:25.911112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:26.007067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1067/2114] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:25 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:26.012066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:26.022463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc50d2-4d81-403e-bc70-72c4168a5863 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:26.081440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905a8b-d0a5-47ea-98d7-4aa44d1f2506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:26.082212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905a8b-d0a5-47ea-98d7-4aa44d1f2506 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.082501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905a8b-d0a5-47ea-98d7-4aa44d1f2506 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.082782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3905a8b-d0a5-47ea-98d7-4aa44d1f2506 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.083503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905a8b-d0a5-47ea-98d7-4aa44d1f2506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:26.085531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3905a8b-d0a5-47ea-98d7-4aa44d1f2506 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:26.177238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1048/2115] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:26 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:26.181948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05eaa1-dd86-427c-84a7-ed51d610e641 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:26.197430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05eaa1-dd86-427c-84a7-ed51d610e641 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:26.203037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd846-a7c3-4cfb-b2e9-48ec1a82f526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:26.203783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd846-a7c3-4cfb-b2e9-48ec1a82f526 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.204072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd846-a7c3-4cfb-b2e9-48ec1a82f526 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.204383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd846-a7c3-4cfb-b2e9-48ec1a82f526 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.205122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd846-a7c3-4cfb-b2e9-48ec1a82f526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:26.206735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd846-a7c3-4cfb-b2e9-48ec1a82f526 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:26.218516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1068/2116] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:26.222940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905a8b-d0a5-47ea-98d7-4aa44d1f2506 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:26.233884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3905a8b-d0a5-47ea-98d7-4aa44d1f2506 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:26.238649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9898898-914d-4f02-abbe-898349cf1c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:26.239340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9898898-914d-4f02-abbe-898349cf1c7a admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.239620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9898898-914d-4f02-abbe-898349cf1c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.239894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9898898-914d-4f02-abbe-898349cf1c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.240677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9898898-914d-4f02-abbe-898349cf1c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:26.242267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9898898-914d-4f02-abbe-898349cf1c7a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:26.253263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1049/2117] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:26.259918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd846-a7c3-4cfb-b2e9-48ec1a82f526 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:26.270497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd846-a7c3-4cfb-b2e9-48ec1a82f526 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:26.275420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952b60-5bce-4ef0-ab71-d7cf876c6637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:26.276819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952b60-5bce-4ef0-ab71-d7cf876c6637 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.277144 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952b60-5bce-4ef0-ab71-d7cf876c6637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.277428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952b60-5bce-4ef0-ab71-d7cf876c6637 admin admin] PATH_INFO: `/v3/projects/425e158ab9b34138b9d65e73801e358e/users/9b41b38a470f47828d24411ba79ba89b/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.295119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46952b60-5bce-4ef0-ab71-d7cf876c6637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425e158ab9b34138b9d65e73801e358e, user_id=9b41b38a470f47828d24411ba79ba89b, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:26.296154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46952b60-5bce-4ef0-ab71-d7cf876c6637 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:26.418152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1069/2118] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:26 2022] PUT /identity/v3/projects/425e158ab9b34138b9d65e73801e358e/users/9b41b38a470f47828d24411ba79ba89b/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:26.429182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a5e5-599a-4b43-8f6f-8920f136adec None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.429404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a5e5-599a-4b43-8f6f-8920f136adec None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.429685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7a5e5-599a-4b43-8f6f-8920f136adec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.463187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-47b7a5e5-599a-4b43-8f6f-8920f136adec None None] MFA Rules not processed for user `9b41b38a470f47828d24411ba79ba89b`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:26.465190 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-47b7a5e5-599a-4b43-8f6f-8920f136adec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:26.556946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1050/2119] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:26.563752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952b60-5bce-4ef0-ab71-d7cf876c6637 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:26.634407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952b60-5bce-4ef0-ab71-d7cf876c6637 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:26.640290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e69b2f-d83b-4a07-bbe4-3bad4db123fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:26.645439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e69b2f-d83b-4a07-bbe4-3bad4db123fd admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.645439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e69b2f-d83b-4a07-bbe4-3bad4db123fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.645439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e69b2f-d83b-4a07-bbe4-3bad4db123fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.645439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e69b2f-d83b-4a07-bbe4-3bad4db123fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:26.645439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e69b2f-d83b-4a07-bbe4-3bad4db123fd admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:26.729424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1070/2120] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:26 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:26.731073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7a5e5-599a-4b43-8f6f-8920f136adec None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:26.741883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7a5e5-599a-4b43-8f6f-8920f136adec None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:26.789324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:26.799676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b3932-f71f-4c47-9145-aa28cf5bb3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:26.800433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b3932-f71f-4c47-9145-aa28cf5bb3b1 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.800666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b3932-f71f-4c47-9145-aa28cf5bb3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.800943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b3932-f71f-4c47-9145-aa28cf5bb3b1 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.801650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b3932-f71f-4c47-9145-aa28cf5bb3b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:26.803769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b3932-f71f-4c47-9145-aa28cf5bb3b1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:26.904185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1051/2121] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:26 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:26.907846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e69b2f-d83b-4a07-bbe4-3bad4db123fd admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:26.918503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e69b2f-d83b-4a07-bbe4-3bad4db123fd admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:26.923124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ad3f8-acf0-4fda-8aff-662ccf55d20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:26.923805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ad3f8-acf0-4fda-8aff-662ccf55d20f admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.924203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ad3f8-acf0-4fda-8aff-662ccf55d20f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.924484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7ad3f8-acf0-4fda-8aff-662ccf55d20f admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.925244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ad3f8-acf0-4fda-8aff-662ccf55d20f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:26.926849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7ad3f8-acf0-4fda-8aff-662ccf55d20f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:26.938627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1071/2122] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:26.942982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b3932-f71f-4c47-9145-aa28cf5bb3b1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:26.954802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b3932-f71f-4c47-9145-aa28cf5bb3b1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:26.960605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7b337-4408-4b23-a5ab-06d079579580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:26.961519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7b337-4408-4b23-a5ab-06d079579580 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:26.962987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7b337-4408-4b23-a5ab-06d079579580 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:26.962987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed7b337-4408-4b23-a5ab-06d079579580 admin admin] PATH_INFO: `/v3/projects/6ed9184cb81145d09b3e905ee600939a/users/d006601532e842578c0e6dae6585092c/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:26.977988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7b337-4408-4b23-a5ab-06d079579580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed9184cb81145d09b3e905ee600939a, user_id=d006601532e842578c0e6dae6585092c, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:26.981022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed7b337-4408-4b23-a5ab-06d079579580 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:27.076461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1052/2123] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:26 2022] PUT /identity/v3/projects/6ed9184cb81145d09b3e905ee600939a/users/d006601532e842578c0e6dae6585092c/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:27.085965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ad3f8-acf0-4fda-8aff-662ccf55d20f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:27.092244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7ad3f8-acf0-4fda-8aff-662ccf55d20f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:27.165541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97bae6-387f-4b31-b7ff-d951b39bd4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:27.166229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97bae6-387f-4b31-b7ff-d951b39bd4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:27.166537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97bae6-387f-4b31-b7ff-d951b39bd4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:27.166821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97bae6-387f-4b31-b7ff-d951b39bd4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:27.167565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97bae6-387f-4b31-b7ff-d951b39bd4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:27.169280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97bae6-387f-4b31-b7ff-d951b39bd4b0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:27.179389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1072/2124] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:27.184036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7b337-4408-4b23-a5ab-06d079579580 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:27.194688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed7b337-4408-4b23-a5ab-06d079579580 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:27.200372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:27.201300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:27.201604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:27.201854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] PATH_INFO: `/v3/projects/6ed9184cb81145d09b3e905ee600939a/users/d006601532e842578c0e6dae6585092c/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:27.203928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed9184cb81145d09b3e905ee600939a, user_id=d006601532e842578c0e6dae6585092c, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:27.206660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:27.218283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1053/2125] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:27 2022] PUT /identity/v3/projects/6ed9184cb81145d09b3e905ee600939a/users/d006601532e842578c0e6dae6585092c/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:27.228670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feb824-bac4-462b-99d3-af0cd201420b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:27.228955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feb824-bac4-462b-99d3-af0cd201420b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:27.229473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0feb824-bac4-462b-99d3-af0cd201420b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:27.256223 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f0feb824-bac4-462b-99d3-af0cd201420b None None] MFA Rules not processed for user `d006601532e842578c0e6dae6585092c`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:27.257456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f0feb824-bac4-462b-99d3-af0cd201420b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:27.272817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:27.337653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1073/2126] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:27.349428 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:27.351270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:27.421152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6879b21-9b99-48f6-9d14-20040389057d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:27.425040 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae894b48-c9b9-4556-baca-f653b1b9524f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:27.425732 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae894b48-c9b9-4556-baca-f653b1b9524f service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:27.426012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae894b48-c9b9-4556-baca-f653b1b9524f service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:27.426285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae894b48-c9b9-4556-baca-f653b1b9524f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:27.436329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae894b48-c9b9-4556-baca-f653b1b9524f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:27.437758 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae894b48-c9b9-4556-baca-f653b1b9524f service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:27.447082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1054/2127] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:27.524404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feb824-bac4-462b-99d3-af0cd201420b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:27.611629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0feb824-bac4-462b-99d3-af0cd201420b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:27.617235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:27.618227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:27.618514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:27.618788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] PATH_INFO: `/v3/users/9b41b38a470f47828d24411ba79ba89b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:27.621599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b41b38a470f47828d24411ba79ba89b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:27.623616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:27.809959 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b41b38a470f47828d24411ba79ba89b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:27.897110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b41b38a470f47828d24411ba79ba89b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:27.897732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b41b38a470f47828d24411ba79ba89b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:27.906217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b41b38a470f47828d24411ba79ba89b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:27.906944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b41b38a470f47828d24411ba79ba89b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:27.926514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1074/2128] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:27 2022] DELETE /identity/v3/users/9b41b38a470f47828d24411ba79ba89b => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:27.931188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee548-e540-4107-9cdd-28a5eaef5067 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:27.931706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee548-e540-4107-9cdd-28a5eaef5067 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:27.931869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ee548-e540-4107-9cdd-28a5eaef5067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:27.953402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4d7ee548-e540-4107-9cdd-28a5eaef5067 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:27.954626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4d7ee548-e540-4107-9cdd-28a5eaef5067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:28.023396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1055/2129] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:28.039471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:28.041281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:28.081351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:28.112051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510fc83-6bb4-4960-beff-fe5ad418e890 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:28.115961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a027d-b11c-4521-a81b-5b78210cf41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:28.117225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a027d-b11c-4521-a81b-5b78210cf41a service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:28.117509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a027d-b11c-4521-a81b-5b78210cf41a service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:28.117785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a027d-b11c-4521-a81b-5b78210cf41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:28.129136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a027d-b11c-4521-a81b-5b78210cf41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:28.130620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a027d-b11c-4521-a81b-5b78210cf41a service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:28.148350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1075/2130] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:28.689288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ee548-e540-4107-9cdd-28a5eaef5067 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:28.690736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4d7ee548-e540-4107-9cdd-28a5eaef5067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:28.705756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ee548-e540-4107-9cdd-28a5eaef5067 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:28.719588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:28.719588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:28.719588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:28.719588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] PATH_INFO: `/v3/projects/425e158ab9b34138b9d65e73801e358e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:28.719914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=425e158ab9b34138b9d65e73801e358e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:28.721562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:28.849769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:28.856424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425e158ab9b34138b9d65e73801e358e', 'request_id': 'req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:28.964213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425e158ab9b34138b9d65e73801e358e', 'request_id': 'req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:28.970333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425e158ab9b34138b9d65e73801e358e', 'request_id': 'req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:28.983822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] The token cache is being invalidate because project 425e158ab9b34138b9d65e73801e358e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:28.984376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:28.985061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:28.986943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1056/2131] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:28 2022] DELETE /identity/v3/projects/425e158ab9b34138b9d65e73801e358e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:28.991794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a027d-b11c-4521-a81b-5b78210cf41a service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:29.004998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a027d-b11c-4521-a81b-5b78210cf41a service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:29.078941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:29.079837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:29.080256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:29.080553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] PATH_INFO: `/v3/users/d006601532e842578c0e6dae6585092c` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:29.081780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d006601532e842578c0e6dae6585092c)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:29.083742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:29.417563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd006601532e842578c0e6dae6585092c'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:29.474244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd006601532e842578c0e6dae6585092c'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:29.474905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd006601532e842578c0e6dae6585092c'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:29.515357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd006601532e842578c0e6dae6585092c'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:29.516482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a6912080-2f69-4ea2-9047-1524b06170d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd006601532e842578c0e6dae6585092c'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:29.559494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1076/2132] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:28 2022] DELETE /identity/v3/users/d006601532e842578c0e6dae6585092c => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:30.183859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:30.185376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:30.200570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba7d9d-a8ef-4bb9-8c20-65c097a54bae admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:30.278114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:30.278997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:30.279284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:30.281374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] PATH_INFO: `/v3/projects/6ed9184cb81145d09b3e905ee600939a` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:30.281374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed9184cb81145d09b3e905ee600939a)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:30.282804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:30.343698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edb452-e6be-4fac-b1e9-cbe6685141cf None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:30.344282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edb452-e6be-4fac-b1e9-cbe6685141cf None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:30.344387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edb452-e6be-4fac-b1e9-cbe6685141cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:30.362899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-49edb452-e6be-4fac-b1e9-cbe6685141cf None None] MFA Rules not processed for user `62e566aa74b045cb8cc6e8edf4a256e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:30.364094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-49edb452-e6be-4fac-b1e9-cbe6685141cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:30.411148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:30.421227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1077/2133] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:30.437537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed9184cb81145d09b3e905ee600939a', 'request_id': 'req-8c636e07-d271-4781-9fe2-f7ef4b72110c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:30.529541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed9184cb81145d09b3e905ee600939a', 'request_id': 'req-8c636e07-d271-4781-9fe2-f7ef4b72110c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:30.536150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed9184cb81145d09b3e905ee600939a', 'request_id': 'req-8c636e07-d271-4781-9fe2-f7ef4b72110c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:30.551115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] The token cache is being invalidate because project 6ed9184cb81145d09b3e905ee600939a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:30.551408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:30.552311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-8c636e07-d271-4781-9fe2-f7ef4b72110c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:30.554088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1057/2134] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:30 2022] DELETE /identity/v3/projects/6ed9184cb81145d09b3e905ee600939a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:30.559251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617eff1-5b66-414b-a0c0-7a5470ca33d8 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:30.559632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617eff1-5b66-414b-a0c0-7a5470ca33d8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:30.562146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617eff1-5b66-414b-a0c0-7a5470ca33d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:30.585313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1617eff1-5b66-414b-a0c0-7a5470ca33d8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:30.585313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1617eff1-5b66-414b-a0c0-7a5470ca33d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:30.642794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1078/2135] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:30.652551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f8d53-4f9f-4cb0-b09d-9ffdf26c3d11 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:30.652844 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f8d53-4f9f-4cb0-b09d-9ffdf26c3d11 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:30.653126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13f8d53-4f9f-4cb0-b09d-9ffdf26c3d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:30.673741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e13f8d53-4f9f-4cb0-b09d-9ffdf26c3d11 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:30.678828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e13f8d53-4f9f-4cb0-b09d-9ffdf26c3d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:30.684032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1058/2136] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:30.688093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617eff1-5b66-414b-a0c0-7a5470ca33d8 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:30.698475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617eff1-5b66-414b-a0c0-7a5470ca33d8 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:30.703354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:30.705594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:30.705594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:30.705594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:30.705594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:30.705594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:30.707169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:30.719505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1079/2137] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:30.724066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f8d53-4f9f-4cb0-b09d-9ffdf26c3d11 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:30.734138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13f8d53-4f9f-4cb0-b09d-9ffdf26c3d11 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:30.738846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e370f1-0d5a-493f-a0b8-2d32a5ad895f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:30.739520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e370f1-0d5a-493f-a0b8-2d32a5ad895f admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:30.739786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e370f1-0d5a-493f-a0b8-2d32a5ad895f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:30.740069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e370f1-0d5a-493f-a0b8-2d32a5ad895f admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:30.740838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e370f1-0d5a-493f-a0b8-2d32a5ad895f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:30.742723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e370f1-0d5a-493f-a0b8-2d32a5ad895f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:30.859943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1059/2138] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:30 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:30.863404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:30.873376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9149f-6f4c-4710-a206-f29c51ecd9d6 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:30.933775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abbd2e-c716-4026-99a9-ea2aef51d0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:30.934489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abbd2e-c716-4026-99a9-ea2aef51d0af admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:30.934769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abbd2e-c716-4026-99a9-ea2aef51d0af admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:30.935057 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abbd2e-c716-4026-99a9-ea2aef51d0af admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:30.935922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abbd2e-c716-4026-99a9-ea2aef51d0af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:30.938125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abbd2e-c716-4026-99a9-ea2aef51d0af admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.023344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1080/2139] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:30 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:31.027619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e370f1-0d5a-493f-a0b8-2d32a5ad895f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.038019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e370f1-0d5a-493f-a0b8-2d32a5ad895f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.042821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c66d3-5796-4761-8cab-e78e21b11d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.043480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c66d3-5796-4761-8cab-e78e21b11d46 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.043762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c66d3-5796-4761-8cab-e78e21b11d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.044087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c66d3-5796-4761-8cab-e78e21b11d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.044840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c66d3-5796-4761-8cab-e78e21b11d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.046453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c66d3-5796-4761-8cab-e78e21b11d46 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.057649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1060/2140] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:31.061983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abbd2e-c716-4026-99a9-ea2aef51d0af admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.071891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abbd2e-c716-4026-99a9-ea2aef51d0af admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.077490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84db624-c8ba-4e9a-bc88-ae7a7612e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.078189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84db624-c8ba-4e9a-bc88-ae7a7612e596 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.078478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84db624-c8ba-4e9a-bc88-ae7a7612e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.078758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84db624-c8ba-4e9a-bc88-ae7a7612e596 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.079494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84db624-c8ba-4e9a-bc88-ae7a7612e596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.081170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84db624-c8ba-4e9a-bc88-ae7a7612e596 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.092971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1081/2141] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:31.097065 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c66d3-5796-4761-8cab-e78e21b11d46 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.107452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c66d3-5796-4761-8cab-e78e21b11d46 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.112070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8563-8a5d-4caf-be00-f0f7db6266de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.113017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8563-8a5d-4caf-be00-f0f7db6266de admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.113596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8563-8a5d-4caf-be00-f0f7db6266de admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.113878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb8563-8a5d-4caf-be00-f0f7db6266de admin admin] PATH_INFO: `/v3/projects/190062aa29794f4aba5d45e2d7ce4ce7/users/a36bed3906c5479bb60247c61615c19c/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.132630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb8563-8a5d-4caf-be00-f0f7db6266de admin admin] RBAC: Authorizing `identity:create_grant(project_id=190062aa29794f4aba5d45e2d7ce4ce7, user_id=a36bed3906c5479bb60247c61615c19c, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.141641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb8563-8a5d-4caf-be00-f0f7db6266de admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.141641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:31.211127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1061/2142] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:31 2022] PUT /identity/v3/projects/190062aa29794f4aba5d45e2d7ce4ce7/users/a36bed3906c5479bb60247c61615c19c/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:31.221669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a922fe-7e61-411d-b54f-1c062095e622 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.221969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a922fe-7e61-411d-b54f-1c062095e622 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.222257 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a922fe-7e61-411d-b54f-1c062095e622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.249270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-60a922fe-7e61-411d-b54f-1c062095e622 None None] MFA Rules not processed for user `a36bed3906c5479bb60247c61615c19c`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:31.250496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-60a922fe-7e61-411d-b54f-1c062095e622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:31.308795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:31.318534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1082/2143] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:31.323552 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8563-8a5d-4caf-be00-f0f7db6266de admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.387097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb8563-8a5d-4caf-be00-f0f7db6266de admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.392095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b30942-fd7e-4f0d-9c36-d4423dcb6631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.392781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b30942-fd7e-4f0d-9c36-d4423dcb6631 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.393060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b30942-fd7e-4f0d-9c36-d4423dcb6631 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.393337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b30942-fd7e-4f0d-9c36-d4423dcb6631 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.394014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b30942-fd7e-4f0d-9c36-d4423dcb6631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.395918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b30942-fd7e-4f0d-9c36-d4423dcb6631 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.484762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1062/2144] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:31 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:31.485906 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a922fe-7e61-411d-b54f-1c062095e622 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.500009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a922fe-7e61-411d-b54f-1c062095e622 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.558898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1e148-2d3e-4716-a1b9-47c0df89f26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.559477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1e148-2d3e-4716-a1b9-47c0df89f26d admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.559759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1e148-2d3e-4716-a1b9-47c0df89f26d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.560036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1e148-2d3e-4716-a1b9-47c0df89f26d admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.560749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1e148-2d3e-4716-a1b9-47c0df89f26d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.562662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1e148-2d3e-4716-a1b9-47c0df89f26d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.643978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1083/2145] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:31 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:31.648249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b30942-fd7e-4f0d-9c36-d4423dcb6631 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.658805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b30942-fd7e-4f0d-9c36-d4423dcb6631 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.663631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36d1f2-80aa-42ca-955a-d428c09eb8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.664348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d1f2-80aa-42ca-955a-d428c09eb8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.664616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d1f2-80aa-42ca-955a-d428c09eb8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.664913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d1f2-80aa-42ca-955a-d428c09eb8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.667017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36d1f2-80aa-42ca-955a-d428c09eb8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.668497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36d1f2-80aa-42ca-955a-d428c09eb8b1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.679021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1063/2146] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:31.683848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1e148-2d3e-4716-a1b9-47c0df89f26d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.694103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1e148-2d3e-4716-a1b9-47c0df89f26d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.698824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fe67b-45f6-4d47-b2b7-23f4b69f9c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.699725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe67b-45f6-4d47-b2b7-23f4b69f9c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.699992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe67b-45f6-4d47-b2b7-23f4b69f9c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.700306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fe67b-45f6-4d47-b2b7-23f4b69f9c15 admin admin] PATH_INFO: `/v3/projects/2e3534e04fba41c9a554ea56fcea9111/users/29dc3f7f0c3742799c1750212377cf8e/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.715588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fe67b-45f6-4d47-b2b7-23f4b69f9c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3534e04fba41c9a554ea56fcea9111, user_id=29dc3f7f0c3742799c1750212377cf8e, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.718306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fe67b-45f6-4d47-b2b7-23f4b69f9c15 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.789629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1084/2147] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:31 2022] PUT /identity/v3/projects/2e3534e04fba41c9a554ea56fcea9111/users/29dc3f7f0c3742799c1750212377cf8e/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:31.794247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36d1f2-80aa-42ca-955a-d428c09eb8b1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.804149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36d1f2-80aa-42ca-955a-d428c09eb8b1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.868842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73888932-0d5d-4299-a79f-6bdf08d9377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.869574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888932-0d5d-4299-a79f-6bdf08d9377b admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.869865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888932-0d5d-4299-a79f-6bdf08d9377b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.870145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73888932-0d5d-4299-a79f-6bdf08d9377b admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.870891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73888932-0d5d-4299-a79f-6bdf08d9377b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.872757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73888932-0d5d-4299-a79f-6bdf08d9377b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.884236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1064/2148] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:31.888382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fe67b-45f6-4d47-b2b7-23f4b69f9c15 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:31.899122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fe67b-45f6-4d47-b2b7-23f4b69f9c15 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:31.904988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:31.905868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.906185 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.906459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] PATH_INFO: `/v3/projects/2e3534e04fba41c9a554ea56fcea9111/users/29dc3f7f0c3742799c1750212377cf8e/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.908495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e3534e04fba41c9a554ea56fcea9111, user_id=29dc3f7f0c3742799c1750212377cf8e, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:31.911118 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:31.922344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1085/2149] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:31 2022] PUT /identity/v3/projects/2e3534e04fba41c9a554ea56fcea9111/users/29dc3f7f0c3742799c1750212377cf8e/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:31.938277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8963a9-efd5-4ad9-8ce7-8d781fc5cfe2 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:31.938575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8963a9-efd5-4ad9-8ce7-8d781fc5cfe2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:31.938861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8963a9-efd5-4ad9-8ce7-8d781fc5cfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:31.966084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-fe8963a9-efd5-4ad9-8ce7-8d781fc5cfe2 None None] MFA Rules not processed for user `29dc3f7f0c3742799c1750212377cf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:31.967639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fe8963a9-efd5-4ad9-8ce7-8d781fc5cfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:32.025164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:32.043356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1065/2150] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:32.054371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:32.056135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:32.115404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ad35a-b265-4bc9-9715-4048ba4ba3cd admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:32.119484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ade297-5080-4551-97cc-a231f72d1415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:32.120713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade297-5080-4551-97cc-a231f72d1415 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:32.121002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade297-5080-4551-97cc-a231f72d1415 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:32.121327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ade297-5080-4551-97cc-a231f72d1415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:32.131245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ade297-5080-4551-97cc-a231f72d1415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:32.132727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ade297-5080-4551-97cc-a231f72d1415 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:32.143580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1086/2151] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:33.288757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8963a9-efd5-4ad9-8ce7-8d781fc5cfe2 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:33.351592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8963a9-efd5-4ad9-8ce7-8d781fc5cfe2 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:33.357283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:33.358296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:33.358627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:33.358961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] PATH_INFO: `/v3/users/a36bed3906c5479bb60247c61615c19c` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:33.360433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a36bed3906c5479bb60247c61615c19c)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:33.362724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:33.515444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a36bed3906c5479bb60247c61615c19c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:33.599459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a36bed3906c5479bb60247c61615c19c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:33.600105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a36bed3906c5479bb60247c61615c19c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:33.608319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a36bed3906c5479bb60247c61615c19c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:33.608967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a36bed3906c5479bb60247c61615c19c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:33.627822 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1066/2152] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:33 2022] DELETE /identity/v3/users/a36bed3906c5479bb60247c61615c19c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:33.632818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae116cd-3003-4223-af9a-4d7b24261b93 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:33.633343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae116cd-3003-4223-af9a-4d7b24261b93 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:33.633472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae116cd-3003-4223-af9a-4d7b24261b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:33.655463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0ae116cd-3003-4223-af9a-4d7b24261b93 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:33.656710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0ae116cd-3003-4223-af9a-4d7b24261b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:33.701113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:33.716071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1087/2153] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:33.725725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:33.727251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:33.789371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc13809-e78a-4e9c-a794-4320db1d9e42 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:33.793897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b496b-368c-4a57-8fb3-cb981081a331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:33.794661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b496b-368c-4a57-8fb3-cb981081a331 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:33.794988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b496b-368c-4a57-8fb3-cb981081a331 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:33.795308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b496b-368c-4a57-8fb3-cb981081a331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:33.805333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b496b-368c-4a57-8fb3-cb981081a331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:33.806764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b496b-368c-4a57-8fb3-cb981081a331 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:33.817247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1067/2154] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:34.218722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae116cd-3003-4223-af9a-4d7b24261b93 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:34.220353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0ae116cd-3003-4223-af9a-4d7b24261b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:34.247815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae116cd-3003-4223-af9a-4d7b24261b93 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:34.253644 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:34.254542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.254877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.255129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] PATH_INFO: `/v3/projects/190062aa29794f4aba5d45e2d7ce4ce7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.256449 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=190062aa29794f4aba5d45e2d7ce4ce7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:34.258365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:34.356277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190062aa29794f4aba5d45e2d7ce4ce7', 'request_id': 'req-a5473c93-dc7a-4bc9-995f-03bd8419ea81'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.398673 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190062aa29794f4aba5d45e2d7ce4ce7', 'request_id': 'req-a5473c93-dc7a-4bc9-995f-03bd8419ea81'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.405296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190062aa29794f4aba5d45e2d7ce4ce7', 'request_id': 'req-a5473c93-dc7a-4bc9-995f-03bd8419ea81'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.416139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] The token cache is being invalidate because project 190062aa29794f4aba5d45e2d7ce4ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:34.416435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.417047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.418931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1088/2155] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:34 2022] DELETE /identity/v3/projects/190062aa29794f4aba5d45e2d7ce4ce7 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:34.422902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b496b-368c-4a57-8fb3-cb981081a331 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:34.432716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b496b-368c-4a57-8fb3-cb981081a331 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:34.441932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:34.492773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:34.493649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:34.493931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:34.494204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] PATH_INFO: `/v3/users/29dc3f7f0c3742799c1750212377cf8e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:34.495464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29dc3f7f0c3742799c1750212377cf8e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:34.497768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:34.722741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29dc3f7f0c3742799c1750212377cf8e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.821959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29dc3f7f0c3742799c1750212377cf8e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.822593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29dc3f7f0c3742799c1750212377cf8e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.830677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29dc3f7f0c3742799c1750212377cf8e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.831338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-97f0d703-7949-4007-bef7-e409c00b5ec1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29dc3f7f0c3742799c1750212377cf8e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:34.850282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1068/2156] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:34 2022] DELETE /identity/v3/users/29dc3f7f0c3742799c1750212377cf8e => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:35.282874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:35.284497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:35.298766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5473c93-dc7a-4bc9-995f-03bd8419ea81 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:35.359398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:35.360305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:35.360595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:35.360873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] PATH_INFO: `/v3/projects/2e3534e04fba41c9a554ea56fcea9111` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:35.362066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e3534e04fba41c9a554ea56fcea9111)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:35.363992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:35.451698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e3534e04fba41c9a554ea56fcea9111', 'request_id': 'req-742d4567-b7c9-452e-9d54-4a3abb0f5a25'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.517414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e3534e04fba41c9a554ea56fcea9111', 'request_id': 'req-742d4567-b7c9-452e-9d54-4a3abb0f5a25'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.524681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e3534e04fba41c9a554ea56fcea9111', 'request_id': 'req-742d4567-b7c9-452e-9d54-4a3abb0f5a25'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.535664 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] The token cache is being invalidate because project 2e3534e04fba41c9a554ea56fcea9111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:35.535957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.536631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-742d4567-b7c9-452e-9d54-4a3abb0f5a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:35.538170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1089/2157] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:35 2022] DELETE /identity/v3/projects/2e3534e04fba41c9a554ea56fcea9111 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:35.543349 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa032e3-cdbe-430d-9b26-44fdc2750abf None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:35.543963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa032e3-cdbe-430d-9b26-44fdc2750abf None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:35.543963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa032e3-cdbe-430d-9b26-44fdc2750abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:35.563776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-baa032e3-cdbe-430d-9b26-44fdc2750abf None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:35.565060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-baa032e3-cdbe-430d-9b26-44fdc2750abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:35.623884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1069/2158] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:35.632010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbfff4-7d2c-467b-b942-6b1ffebd371f None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:35.635385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbfff4-7d2c-467b-b942-6b1ffebd371f None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:35.635385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbfff4-7d2c-467b-b942-6b1ffebd371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:35.640104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:35.653683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9ddbfff4-7d2c-467b-b942-6b1ffebd371f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:35.654896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9ddbfff4-7d2c-467b-b942-6b1ffebd371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:35.662964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1090/2159] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:35.667358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa032e3-cdbe-430d-9b26-44fdc2750abf None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:35.677301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa032e3-cdbe-430d-9b26-44fdc2750abf None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:35.681985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:35.682670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:35.682961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:35.683269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:35.684747 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:35.685061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:35.686686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:35.698936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1070/2160] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:35.710508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbfff4-7d2c-467b-b942-6b1ffebd371f None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:35.720151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbfff4-7d2c-467b-b942-6b1ffebd371f None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:35.726454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c4baa-5948-49d3-92ca-1f17df762be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:35.727155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c4baa-5948-49d3-92ca-1f17df762be3 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:35.727444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c4baa-5948-49d3-92ca-1f17df762be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:35.727722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c4baa-5948-49d3-92ca-1f17df762be3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:35.728462 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c4baa-5948-49d3-92ca-1f17df762be3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:35.730447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c4baa-5948-49d3-92ca-1f17df762be3 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:35.870928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1091/2161] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:35 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:35.874865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:35.885116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeda39-c76f-4e85-9621-e6acd9904df1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:35.904179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:35.944965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258583ac-0fd5-4e80-b4a4-aec7e8ebe5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:35.945678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258583ac-0fd5-4e80-b4a4-aec7e8ebe5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:35.945958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258583ac-0fd5-4e80-b4a4-aec7e8ebe5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:35.946234 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258583ac-0fd5-4e80-b4a4-aec7e8ebe5d1 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:35.946913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258583ac-0fd5-4e80-b4a4-aec7e8ebe5d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:35.948846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258583ac-0fd5-4e80-b4a4-aec7e8ebe5d1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.033146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1071/2162] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:35 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:36.037238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c4baa-5948-49d3-92ca-1f17df762be3 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.048329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c4baa-5948-49d3-92ca-1f17df762be3 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.053263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebea4c-b55e-4080-8d4f-a1e9b32b608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.053932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebea4c-b55e-4080-8d4f-a1e9b32b608e admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.054215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebea4c-b55e-4080-8d4f-a1e9b32b608e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.054492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebea4c-b55e-4080-8d4f-a1e9b32b608e admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.055229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebea4c-b55e-4080-8d4f-a1e9b32b608e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.056895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebea4c-b55e-4080-8d4f-a1e9b32b608e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.067773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1092/2163] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:36.072675 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258583ac-0fd5-4e80-b4a4-aec7e8ebe5d1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.083308 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258583ac-0fd5-4e80-b4a4-aec7e8ebe5d1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.088214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe898129-bd9d-4eae-a2cd-097c2630bf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.088875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe898129-bd9d-4eae-a2cd-097c2630bf20 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.089167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe898129-bd9d-4eae-a2cd-097c2630bf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.093037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe898129-bd9d-4eae-a2cd-097c2630bf20 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.093037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe898129-bd9d-4eae-a2cd-097c2630bf20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.093037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe898129-bd9d-4eae-a2cd-097c2630bf20 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.103839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1072/2164] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:36.109013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebea4c-b55e-4080-8d4f-a1e9b32b608e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.121872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebea4c-b55e-4080-8d4f-a1e9b32b608e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.124634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97458a89-442a-4af4-8631-02eaa373e4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.125523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97458a89-442a-4af4-8631-02eaa373e4d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.125811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97458a89-442a-4af4-8631-02eaa373e4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.126091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97458a89-442a-4af4-8631-02eaa373e4d8 admin admin] PATH_INFO: `/v3/projects/2d4099ae8f3d4dd9bd461aab44a9f0e9/users/3cff620478be47abbecb6cfd2cec3dec/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.141330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97458a89-442a-4af4-8631-02eaa373e4d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4099ae8f3d4dd9bd461aab44a9f0e9, user_id=3cff620478be47abbecb6cfd2cec3dec, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.144000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97458a89-442a-4af4-8631-02eaa373e4d8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.232495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1093/2165] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:36 2022] PUT /identity/v3/projects/2d4099ae8f3d4dd9bd461aab44a9f0e9/users/3cff620478be47abbecb6cfd2cec3dec/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:36.242816 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8b9a-f129-433b-ac5c-2aaf6c299879 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.243113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8b9a-f129-433b-ac5c-2aaf6c299879 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.243398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8b9a-f129-433b-ac5c-2aaf6c299879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.271800 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-14ff8b9a-f129-433b-ac5c-2aaf6c299879 None None] MFA Rules not processed for user `3cff620478be47abbecb6cfd2cec3dec`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:36.273042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-14ff8b9a-f129-433b-ac5c-2aaf6c299879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:36.338525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1073/2166] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:36.344081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97458a89-442a-4af4-8631-02eaa373e4d8 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.408061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97458a89-442a-4af4-8631-02eaa373e4d8 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.413092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8b6d-705d-49bf-8aac-dff6bb2d8dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.413750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8b6d-705d-49bf-8aac-dff6bb2d8dcd admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.414062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8b6d-705d-49bf-8aac-dff6bb2d8dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.414343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b8b6d-705d-49bf-8aac-dff6bb2d8dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.415029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b8b6d-705d-49bf-8aac-dff6bb2d8dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.416993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b8b6d-705d-49bf-8aac-dff6bb2d8dcd admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.497750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1094/2167] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:36 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:36.501798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff8b9a-f129-433b-ac5c-2aaf6c299879 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.513332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff8b9a-f129-433b-ac5c-2aaf6c299879 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.568519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c55056-3174-4d34-a237-6521b6d717b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.569220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c55056-3174-4d34-a237-6521b6d717b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.569465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c55056-3174-4d34-a237-6521b6d717b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.569784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c55056-3174-4d34-a237-6521b6d717b8 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.570608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c55056-3174-4d34-a237-6521b6d717b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.572571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c55056-3174-4d34-a237-6521b6d717b8 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.633514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1074/2168] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:36 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:36.638049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8b6d-705d-49bf-8aac-dff6bb2d8dcd admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.648733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b8b6d-705d-49bf-8aac-dff6bb2d8dcd admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.654187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadebf4-3d70-49c5-8c28-ec1287fe3c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.654848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadebf4-3d70-49c5-8c28-ec1287fe3c14 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.655128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadebf4-3d70-49c5-8c28-ec1287fe3c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.655401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fadebf4-3d70-49c5-8c28-ec1287fe3c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.657337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadebf4-3d70-49c5-8c28-ec1287fe3c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.657962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fadebf4-3d70-49c5-8c28-ec1287fe3c14 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.668798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1095/2169] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:36.673592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c55056-3174-4d34-a237-6521b6d717b8 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.683523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c55056-3174-4d34-a237-6521b6d717b8 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.688037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00e662-1727-4504-b639-25e83c60a87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.688949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00e662-1727-4504-b639-25e83c60a87c admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.689229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00e662-1727-4504-b639-25e83c60a87c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.689504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00e662-1727-4504-b639-25e83c60a87c admin admin] PATH_INFO: `/v3/projects/b2a89a0644ce4c2393a5b8a261f10223/users/799334acbb5c425b8d8466fc9f71736c/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.705086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00e662-1727-4504-b639-25e83c60a87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a89a0644ce4c2393a5b8a261f10223, user_id=799334acbb5c425b8d8466fc9f71736c, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.707765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00e662-1727-4504-b639-25e83c60a87c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.709989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:36.798214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1075/2170] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:36 2022] PUT /identity/v3/projects/b2a89a0644ce4c2393a5b8a261f10223/users/799334acbb5c425b8d8466fc9f71736c/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:36.802439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadebf4-3d70-49c5-8c28-ec1287fe3c14 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.812754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fadebf4-3d70-49c5-8c28-ec1287fe3c14 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.873678 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379863de-06ad-4aff-9994-5e7826e81d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.874360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379863de-06ad-4aff-9994-5e7826e81d41 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.874647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379863de-06ad-4aff-9994-5e7826e81d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.874932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379863de-06ad-4aff-9994-5e7826e81d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.875674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379863de-06ad-4aff-9994-5e7826e81d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.877336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379863de-06ad-4aff-9994-5e7826e81d41 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.889137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1096/2171] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:01:36.893227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00e662-1727-4504-b639-25e83c60a87c admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:36.908155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00e662-1727-4504-b639-25e83c60a87c admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:36.913443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:36.914419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.914707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.914986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] PATH_INFO: `/v3/projects/b2a89a0644ce4c2393a5b8a261f10223/users/799334acbb5c425b8d8466fc9f71736c/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.917252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a89a0644ce4c2393a5b8a261f10223, user_id=799334acbb5c425b8d8466fc9f71736c, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:36.921207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:36.935704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1076/2172] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:36 2022] PUT /identity/v3/projects/b2a89a0644ce4c2393a5b8a261f10223/users/799334acbb5c425b8d8466fc9f71736c/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:36.947605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152c34-13d2-44ab-bb66-bf47554e96f4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:36.948081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152c34-13d2-44ab-bb66-bf47554e96f4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:36.948538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152c34-13d2-44ab-bb66-bf47554e96f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:36.972845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:36.983275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-12152c34-13d2-44ab-bb66-bf47554e96f4 None None] MFA Rules not processed for user `799334acbb5c425b8d8466fc9f71736c`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:36.984544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-12152c34-13d2-44ab-bb66-bf47554e96f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:37.111782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1097/2173] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:37.123793 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:37.125817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:37.196731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a6424-62f3-404b-bc33-d39cddfc70ee admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:37.204646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13451-32a8-41d0-9e52-6ce7e1e7c680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:37.205320 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13451-32a8-41d0-9e52-6ce7e1e7c680 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:37.205613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13451-32a8-41d0-9e52-6ce7e1e7c680 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:37.205897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13451-32a8-41d0-9e52-6ce7e1e7c680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:37.313772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba13451-32a8-41d0-9e52-6ce7e1e7c680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:37.318598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba13451-32a8-41d0-9e52-6ce7e1e7c680 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:37.334621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1077/2174] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:39.848200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d2181-903f-4d7d-84af-40cc0591f469 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:39.848734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d2181-903f-4d7d-84af-40cc0591f469 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:39.848734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d2181-903f-4d7d-84af-40cc0591f469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:39.867129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fc4d2181-903f-4d7d-84af-40cc0591f469 None None] MFA Rules not processed for user `8eedefd8d94047478663ceab8962d738`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:39.868612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fc4d2181-903f-4d7d-84af-40cc0591f469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:39.940149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1098/2175] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:40.473401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13451-32a8-41d0-9e52-6ce7e1e7c680 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:40.543437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13451-32a8-41d0-9e52-6ce7e1e7c680 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:40.548754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:40.549636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:40.549922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:40.550219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] PATH_INFO: `/v3/users/3cff620478be47abbecb6cfd2cec3dec` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:40.551502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cff620478be47abbecb6cfd2cec3dec)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:40.553486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:40.724811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:40.737116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cff620478be47abbecb6cfd2cec3dec'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:40.833694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cff620478be47abbecb6cfd2cec3dec'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:40.834447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cff620478be47abbecb6cfd2cec3dec'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:40.842183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cff620478be47abbecb6cfd2cec3dec'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:40.842967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cff620478be47abbecb6cfd2cec3dec'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:40.861682 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1078/2176] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:40 2022] DELETE /identity/v3/users/3cff620478be47abbecb6cfd2cec3dec => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:40.866227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d349d-2f26-4324-95f4-59998b19031d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:40.866754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d349d-2f26-4324-95f4-59998b19031d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:40.866847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d349d-2f26-4324-95f4-59998b19031d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:40.885316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-532d349d-2f26-4324-95f4-59998b19031d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:40.887269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-532d349d-2f26-4324-95f4-59998b19031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:40.985050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1099/2177] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:40.994189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:40.996496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:41.112701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecdc8b-7810-48e8-a34b-a87dd19a02e0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:41.116685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061354-4eb7-4a97-a732-e3f321b204ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:41.117501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061354-4eb7-4a97-a732-e3f321b204ba service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:41.117796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061354-4eb7-4a97-a732-e3f321b204ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:41.118077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061354-4eb7-4a97-a732-e3f321b204ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:41.133088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99061354-4eb7-4a97-a732-e3f321b204ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:41.139633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99061354-4eb7-4a97-a732-e3f321b204ba service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:41.156897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1079/2178] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:41.867750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d349d-2f26-4324-95f4-59998b19031d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:41.869413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-532d349d-2f26-4324-95f4-59998b19031d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:41.889705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d349d-2f26-4324-95f4-59998b19031d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:41.901240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:41.910939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:41.911593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:41.912302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] PATH_INFO: `/v3/projects/2d4099ae8f3d4dd9bd461aab44a9f0e9` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:41.914381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d4099ae8f3d4dd9bd461aab44a9f0e9)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:41.916196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:42.054518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:42.064318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4099ae8f3d4dd9bd461aab44a9f0e9', 'request_id': 'req-90fe429b-784e-4827-bb0c-533ed3693244'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.200355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4099ae8f3d4dd9bd461aab44a9f0e9', 'request_id': 'req-90fe429b-784e-4827-bb0c-533ed3693244'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.208164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4099ae8f3d4dd9bd461aab44a9f0e9', 'request_id': 'req-90fe429b-784e-4827-bb0c-533ed3693244'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.222305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] The token cache is being invalidate because project 2d4099ae8f3d4dd9bd461aab44a9f0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:42.222705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.223431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.225333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1100/2179] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:41 2022] DELETE /identity/v3/projects/2d4099ae8f3d4dd9bd461aab44a9f0e9 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:42.229425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061354-4eb7-4a97-a732-e3f321b204ba service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:42.241317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061354-4eb7-4a97-a732-e3f321b204ba service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:42.302103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:42.302985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:42.303332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:42.303677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] PATH_INFO: `/v3/users/799334acbb5c425b8d8466fc9f71736c` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:42.304883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=799334acbb5c425b8d8466fc9f71736c)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:42.307942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:42.574417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '799334acbb5c425b8d8466fc9f71736c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.644995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '799334acbb5c425b8d8466fc9f71736c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.646086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '799334acbb5c425b8d8466fc9f71736c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.652231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:42.657016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '799334acbb5c425b8d8466fc9f71736c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.657773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-dd504cf6-3dd1-4792-a746-ed7242321f70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '799334acbb5c425b8d8466fc9f71736c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:42.684290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1080/2180] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:42 2022] DELETE /identity/v3/users/799334acbb5c425b8d8466fc9f71736c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:43.204761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:43.206301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:43.224224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe429b-784e-4827-bb0c-533ed3693244 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:43.298549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:43.299451 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:43.299740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:43.300171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] PATH_INFO: `/v3/projects/b2a89a0644ce4c2393a5b8a261f10223` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:43.301343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a89a0644ce4c2393a5b8a261f10223)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:43.303559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:43.404101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a89a0644ce4c2393a5b8a261f10223', 'request_id': 'req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:43.484990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a89a0644ce4c2393a5b8a261f10223', 'request_id': 'req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:43.493816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a89a0644ce4c2393a5b8a261f10223', 'request_id': 'req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:43.510242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] The token cache is being invalidate because project b2a89a0644ce4c2393a5b8a261f10223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:43.510540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:43.512145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1b7fc4dd-2ce3-46fa-a76d-90ec197da72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:43.513733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1101/2181] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:43 2022] DELETE /identity/v3/projects/b2a89a0644ce4c2393a5b8a261f10223 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:43.523451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae7163-6ab9-4e61-ae0f-cdd2f40adb5b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:43.524403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae7163-6ab9-4e61-ae0f-cdd2f40adb5b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:43.524692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae7163-6ab9-4e61-ae0f-cdd2f40adb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:43.586385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-60ae7163-6ab9-4e61-ae0f-cdd2f40adb5b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:43.588254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-60ae7163-6ab9-4e61-ae0f-cdd2f40adb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:43.713655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1081/2182] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:43.721727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92c8e-1932-45ad-a63a-94b32d6ff03c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:43.726659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92c8e-1932-45ad-a63a-94b32d6ff03c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:43.726659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe92c8e-1932-45ad-a63a-94b32d6ff03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:43.753025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-abe92c8e-1932-45ad-a63a-94b32d6ff03c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:43.758412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-abe92c8e-1932-45ad-a63a-94b32d6ff03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:43.766765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1102/2183] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:43.771931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae7163-6ab9-4e61-ae0f-cdd2f40adb5b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:43.783149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae7163-6ab9-4e61-ae0f-cdd2f40adb5b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:43.789473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:43.790271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:43.790556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:43.790833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:43.791581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:43.792021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:43.793884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:43.811823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1082/2184] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:43.812083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe92c8e-1932-45ad-a63a-94b32d6ff03c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:43.823142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe92c8e-1932-45ad-a63a-94b32d6ff03c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:43.829115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f88817-5c5b-4441-8c31-9ec97faa21af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:43.829786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88817-5c5b-4441-8c31-9ec97faa21af admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:43.830073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88817-5c5b-4441-8c31-9ec97faa21af admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:43.830351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88817-5c5b-4441-8c31-9ec97faa21af admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:43.831054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f88817-5c5b-4441-8c31-9ec97faa21af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:43.841956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f88817-5c5b-4441-8c31-9ec97faa21af admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:43.924152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1103/2185] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:43 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:43.925677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:43.936470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d930a48-fb7a-4aeb-8100-18ed0c8f08d7 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:44.029017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c118-aff8-467f-aaef-4d6e76ab6fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:44.030826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c118-aff8-467f-aaef-4d6e76ab6fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:44.031281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c118-aff8-467f-aaef-4d6e76ab6fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:44.032315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c118-aff8-467f-aaef-4d6e76ab6fb7 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:44.033084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c118-aff8-467f-aaef-4d6e76ab6fb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:44.035888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5c118-aff8-467f-aaef-4d6e76ab6fb7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:44.190424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1083/2186] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:43 2022] POST /identity/v3/users => generated 287 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:44.191655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f88817-5c5b-4441-8c31-9ec97faa21af admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:44.206587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f88817-5c5b-4441-8c31-9ec97faa21af admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:44.213000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcf6e6-9dbe-4e9c-9ab7-596e4ec4149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:44.213850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf6e6-9dbe-4e9c-9ab7-596e4ec4149a admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:44.214246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf6e6-9dbe-4e9c-9ab7-596e4ec4149a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:44.214720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcf6e6-9dbe-4e9c-9ab7-596e4ec4149a admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:44.216016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcf6e6-9dbe-4e9c-9ab7-596e4ec4149a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:44.217943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbcf6e6-9dbe-4e9c-9ab7-596e4ec4149a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:44.231837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1104/2187] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:44.235531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c118-aff8-467f-aaef-4d6e76ab6fb7 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:44.248348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5c118-aff8-467f-aaef-4d6e76ab6fb7 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:44.255469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5f728-d537-495e-9068-fae3d1759dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:44.256182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5f728-d537-495e-9068-fae3d1759dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:44.256507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5f728-d537-495e-9068-fae3d1759dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:44.256736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5f728-d537-495e-9068-fae3d1759dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:44.257471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5f728-d537-495e-9068-fae3d1759dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:44.259167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5f728-d537-495e-9068-fae3d1759dc2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:44.271303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1084/2188] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:44.275512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcf6e6-9dbe-4e9c-9ab7-596e4ec4149a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:44.293513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbcf6e6-9dbe-4e9c-9ab7-596e4ec4149a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:44.295672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:44.296651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:44.296938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:44.297267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] PATH_INFO: `/v3/projects/e3f62f9df69f47d0ad40154526b95be1/users/66f64211ceb949d398e47bc18e2abbca/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:44.309823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:44.322529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3f62f9df69f47d0ad40154526b95be1, user_id=66f64211ceb949d398e47bc18e2abbca, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:44.322737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:44.444699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1105/2189] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:44 2022] PUT /identity/v3/projects/e3f62f9df69f47d0ad40154526b95be1/users/66f64211ceb949d398e47bc18e2abbca/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:44.450195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a0b6-69eb-46b5-8820-bdfdc0b50db1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:44.450492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a0b6-69eb-46b5-8820-bdfdc0b50db1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:44.450776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7a0b6-69eb-46b5-8820-bdfdc0b50db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:44.470044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e8c7a0b6-69eb-46b5-8820-bdfdc0b50db1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:44.471335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e8c7a0b6-69eb-46b5-8820-bdfdc0b50db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:44.511386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:44.533949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1085/2190] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:44.546924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:44.546924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:44.607343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa02c6-82cf-47b8-b2e9-a346f179bfa5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:44.611064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:44.611727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:44.612010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:44.612322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:44.625191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:44.630532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:44.638040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1106/2191] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:01:46.157877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db012da-111c-402a-bd2a-57de3b588f37 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:46.158344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db012da-111c-402a-bd2a-57de3b588f37 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:46.158431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db012da-111c-402a-bd2a-57de3b588f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:46.204729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-5db012da-111c-402a-bd2a-57de3b588f37 None None] MFA Rules not processed for user `66f64211ceb949d398e47bc18e2abbca`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:46.207142 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5db012da-111c-402a-bd2a-57de3b588f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:46.315324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1086/2192] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:46.326899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:46.328443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:46.387313 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422960a-1846-4353-9a35-3a5cad29e27e service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:46.391316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:46.391977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:46.394594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:46.394594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:46.404829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:46.406590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:46.422396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1107/2193] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:48.306734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48e4c-9c99-42de-ab58-e5aadf291f8a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:48.309057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48e4c-9c99-42de-ab58-e5aadf291f8a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:48.309057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48e4c-9c99-42de-ab58-e5aadf291f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:48.328402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ccf48e4c-9c99-42de-ab58-e5aadf291f8a None None] MFA Rules not processed for user `66f64211ceb949d398e47bc18e2abbca`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:48.329908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ccf48e4c-9c99-42de-ab58-e5aadf291f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:48.339488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1087/2194] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:01:53.884867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:53.887933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:53.906242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9bc7aa-6939-4f58-9ed2-1f2ac40ae322 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:53.911888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:53.913249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:53.913789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:53.914265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] PATH_INFO: `/v3/users/d24564c93fff4923a1e1a5e0399f36a4` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:53.915667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d24564c93fff4923a1e1a5e0399f36a4)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:53.918028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:54.147977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd24564c93fff4923a1e1a5e0399f36a4'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:54.244363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd24564c93fff4923a1e1a5e0399f36a4'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:54.245810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd24564c93fff4923a1e1a5e0399f36a4'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:54.256628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd24564c93fff4923a1e1a5e0399f36a4'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:54.257413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd24564c93fff4923a1e1a5e0399f36a4'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:54.277865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1108/2195] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:01:53 2022] DELETE /identity/v3/users/d24564c93fff4923a1e1a5e0399f36a4 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:54.282176 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5350f62-2168-40bb-a30f-c666fd875a4d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:54.282662 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5350f62-2168-40bb-a30f-c666fd875a4d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:54.282775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5350f62-2168-40bb-a30f-c666fd875a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:54.303200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f5350f62-2168-40bb-a30f-c666fd875a4d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:54.304474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f5350f62-2168-40bb-a30f-c666fd875a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:54.370169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1088/2196] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:54.388483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:54.390169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:54.400984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:54.470767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11ac50a-c332-4a94-95a1-30c544074255 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:54.476214 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38934d-4a9d-416a-a697-7ff1795a216f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:54.476891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38934d-4a9d-416a-a697-7ff1795a216f service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:54.477194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38934d-4a9d-416a-a697-7ff1795a216f service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:54.477621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38934d-4a9d-416a-a697-7ff1795a216f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:54.490449 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38934d-4a9d-416a-a697-7ff1795a216f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:54.492250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38934d-4a9d-416a-a697-7ff1795a216f service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:54.506297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1109/2197] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:01:55.325687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5350f62-2168-40bb-a30f-c666fd875a4d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:55.327123 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f5350f62-2168-40bb-a30f-c666fd875a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:55.339742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5350f62-2168-40bb-a30f-c666fd875a4d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:55.346335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:55.347467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:55.347968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:55.348477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] PATH_INFO: `/v3/projects/49decbc30e5c4ba8bb7cd58d6b997974` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:55.350137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49decbc30e5c4ba8bb7cd58d6b997974)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:55.352414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:55.509100 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49decbc30e5c4ba8bb7cd58d6b997974', 'request_id': 'req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:55.587498 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49decbc30e5c4ba8bb7cd58d6b997974', 'request_id': 'req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:55.593610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:55.598002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49decbc30e5c4ba8bb7cd58d6b997974', 'request_id': 'req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:55.609943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] The token cache is being invalidate because project 49decbc30e5c4ba8bb7cd58d6b997974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:01:55.610582 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:55.611543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-293c93e2-7e5e-4f5a-bf72-23bb3755e1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:01:55.617008 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1089/2198] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:01:55 2022] DELETE /identity/v3/projects/49decbc30e5c4ba8bb7cd58d6b997974 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:01:55.622590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f19432-c674-46e0-b8fb-9ed1ca06caf8 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:55.623523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f19432-c674-46e0-b8fb-9ed1ca06caf8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:55.624175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f19432-c674-46e0-b8fb-9ed1ca06caf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:55.646259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c0f19432-c674-46e0-b8fb-9ed1ca06caf8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:55.647788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c0f19432-c674-46e0-b8fb-9ed1ca06caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:55.736918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1110/2199] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:55.741356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78749730-1ae3-422d-8a88-bf6407e20654 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:55.741723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78749730-1ae3-422d-8a88-bf6407e20654 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:55.742071 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78749730-1ae3-422d-8a88-bf6407e20654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:55.776331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-78749730-1ae3-422d-8a88-bf6407e20654 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:55.778198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-78749730-1ae3-422d-8a88-bf6407e20654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:55.793439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1090/2200] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:55.794648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f19432-c674-46e0-b8fb-9ed1ca06caf8 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:55.808291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f19432-c674-46e0-b8fb-9ed1ca06caf8 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:55.813016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:55.813870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:55.814256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:55.814630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:55.815474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:01:55.815881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:55.817640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:55.836687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1111/2201] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:01:55.837362 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78749730-1ae3-422d-8a88-bf6407e20654 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:55.849285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78749730-1ae3-422d-8a88-bf6407e20654 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:55.856289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57541145-6da4-4eb9-95a3-8e6fb6088314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:55.857145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57541145-6da4-4eb9-95a3-8e6fb6088314 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:55.857263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57541145-6da4-4eb9-95a3-8e6fb6088314 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:55.865537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57541145-6da4-4eb9-95a3-8e6fb6088314 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:55.865537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57541145-6da4-4eb9-95a3-8e6fb6088314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:55.865537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57541145-6da4-4eb9-95a3-8e6fb6088314 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:55.994415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1091/2202] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:01:55 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:01:56.000539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:56.013337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae383e-978c-4c93-872a-f070dde2b0fd admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:56.181890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158b36b-ec2f-4709-bfea-1eaa53654f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:56.182923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158b36b-ec2f-4709-bfea-1eaa53654f80 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:56.183283 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158b36b-ec2f-4709-bfea-1eaa53654f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:56.184174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158b36b-ec2f-4709-bfea-1eaa53654f80 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:56.185121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158b36b-ec2f-4709-bfea-1eaa53654f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:56.187225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158b36b-ec2f-4709-bfea-1eaa53654f80 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:56.303993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1112/2203] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:01:55 2022] POST /identity/v3/users => generated 287 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:01:56.308339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57541145-6da4-4eb9-95a3-8e6fb6088314 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:56.322668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57541145-6da4-4eb9-95a3-8e6fb6088314 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:56.334620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fdc82-55f0-417f-9f83-4834ce5693c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:56.335296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fdc82-55f0-417f-9f83-4834ce5693c0 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:56.335581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fdc82-55f0-417f-9f83-4834ce5693c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:56.335920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fdc82-55f0-417f-9f83-4834ce5693c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:56.336698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fdc82-55f0-417f-9f83-4834ce5693c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:56.338370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fdc82-55f0-417f-9f83-4834ce5693c0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:56.357667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1092/2204] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:56.359548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158b36b-ec2f-4709-bfea-1eaa53654f80 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:56.378703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158b36b-ec2f-4709-bfea-1eaa53654f80 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:56.397240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f9f46-9d42-4b07-a1a5-5ab614cd3627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:56.398263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9f46-9d42-4b07-a1a5-5ab614cd3627 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:56.398717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9f46-9d42-4b07-a1a5-5ab614cd3627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:56.399183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f9f46-9d42-4b07-a1a5-5ab614cd3627 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:56.400134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f9f46-9d42-4b07-a1a5-5ab614cd3627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:56.402641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257f9f46-9d42-4b07-a1a5-5ab614cd3627 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:56.433256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1113/2205] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:01:56.438729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fdc82-55f0-417f-9f83-4834ce5693c0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:56.457549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fdc82-55f0-417f-9f83-4834ce5693c0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:56.468589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:56.469985 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:56.470275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:56.470570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] PATH_INFO: `/v3/projects/f0e6671f7e7e432b92deccdaf094fed4/users/ff863f4f7a2147239fb491097268c9e8/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:56.506210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e6671f7e7e432b92deccdaf094fed4, user_id=ff863f4f7a2147239fb491097268c9e8, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:56.511239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:56.644618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1093/2206] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:01:56 2022] PUT /identity/v3/projects/f0e6671f7e7e432b92deccdaf094fed4/users/ff863f4f7a2147239fb491097268c9e8/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:01:56.658371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240e3c-8412-4ded-b6eb-05e4469d3196 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:56.658839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240e3c-8412-4ded-b6eb-05e4469d3196 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:56.659094 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240e3c-8412-4ded-b6eb-05e4469d3196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:56.689389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-a8240e3c-8412-4ded-b6eb-05e4469d3196 None None] MFA Rules not processed for user `ff863f4f7a2147239fb491097268c9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:01:56.690778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a8240e3c-8412-4ded-b6eb-05e4469d3196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:56.695170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:01:56.768142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1114/2207] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:01:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:01:56.778880 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:01:56.780735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:01:56.842284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c5f19-aabd-4075-af20-c719a63a372f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:01:56.849285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:01:56.849968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:01:56.850247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:01:56.850863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:01:56.861027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:01:56.862535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:01:56.872516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1094/2208] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:01:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:01.644160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6bef-0503-4106-9a9e-32072ad4ec35 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:01.644865 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6bef-0503-4106-9a9e-32072ad4ec35 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:01.644953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d6bef-0503-4106-9a9e-32072ad4ec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:01.663822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-169d6bef-0503-4106-9a9e-32072ad4ec35 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:01.665125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-169d6bef-0503-4106-9a9e-32072ad4ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:01.721503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1115/2209] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:02:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:01.780691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:01.782252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:01.827702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb05297-00dd-4b97-b355-652bb38a2825 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:01.827994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb05297-00dd-4b97-b355-652bb38a2825 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:01.828348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb05297-00dd-4b97-b355-652bb38a2825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:01.845964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-cbb05297-00dd-4b97-b355-652bb38a2825 None None] MFA Rules not processed for user `ff863f4f7a2147239fb491097268c9e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:01.847148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cbb05297-00dd-4b97-b355-652bb38a2825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:01.854591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1116/2210] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:02:01.861270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef710da-a495-44e1-8528-8ce4310650b2 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:01.866539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20ebcb-1e19-4af4-9728-4164684e5ebf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:01.867194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ebcb-1e19-4af4-9728-4164684e5ebf service swift] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:01.867479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ebcb-1e19-4af4-9728-4164684e5ebf service swift] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:01.867758 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20ebcb-1e19-4af4-9728-4164684e5ebf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:01.878995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20ebcb-1e19-4af4-9728-4164684e5ebf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:01.881178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd20ebcb-1e19-4af4-9728-4164684e5ebf service swift] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:01.893680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1095/2211] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:02:11.840445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=53 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:11.887249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:DELETE=1,keystone:SELECT=89,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:26.826902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05298ff4-de61-4b81-8811-cbaafc22a845 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:26.827911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05298ff4-de61-4b81-8811-cbaafc22a845 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:26.828397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05298ff4-de61-4b81-8811-cbaafc22a845 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:26.831538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1117/2212] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 06:02:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:02:26.837728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279e0ea-e822-4550-bb88-7281a84d44f1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:26.838285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279e0ea-e822-4550-bb88-7281a84d44f1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:26.838285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279e0ea-e822-4550-bb88-7281a84d44f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:26.861360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-5279e0ea-e822-4550-bb88-7281a84d44f1 None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:26.862998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5279e0ea-e822-4550-bb88-7281a84d44f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:26.935733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1096/2213] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 06:02:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:02:26.948760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05298ff4-de61-4b81-8811-cbaafc22a845 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:26.952309 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-05298ff4-de61-4b81-8811-cbaafc22a845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:27.012354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05298ff4-de61-4b81-8811-cbaafc22a845 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:27.015914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc1814-5639-4144-a1a6-12866e983fa1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c0883bc26a04de2916c78df342d2023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:27.016683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1814-5639-4144-a1a6-12866e983fa1 service glance] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:27.017018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1814-5639-4144-a1a6-12866e983fa1 service glance] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:27.017278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc1814-5639-4144-a1a6-12866e983fa1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:27.029429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc1814-5639-4144-a1a6-12866e983fa1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:27.030891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc1814-5639-4144-a1a6-12866e983fa1 service glance] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:27.041878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1118/2214] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:27.302066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd098b1d-4911-47f4-9391-0ee79231f959 None None] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:27.302361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd098b1d-4911-47f4-9391-0ee79231f959 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:27.302644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd098b1d-4911-47f4-9391-0ee79231f959 None None] PATH_INFO: `/` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:27.304012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1097/2215] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 06:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:02:27.309554 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1709-011c-4721-8af7-23d7e62124f7 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:27.316979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1709-011c-4721-8af7-23d7e62124f7 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:27.316979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1709-011c-4721-8af7-23d7e62124f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:27.331518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b36b1709-011c-4721-8af7-23d7e62124f7 None None] MFA Rules not processed for user `b1638c28af8240439d061a3ae0ddc38d`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:27.332782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b36b1709-011c-4721-8af7-23d7e62124f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:27.387018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1119/2216] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 06:02:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:27.399741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd098b1d-4911-47f4-9391-0ee79231f959 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:27.401647 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-cd098b1d-4911-47f4-9391-0ee79231f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:27.415655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd098b1d-4911-47f4-9391-0ee79231f959 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:27.419419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15de1c-699c-48c3-8c4d-0b5c2da27293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:27.420077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15de1c-699c-48c3-8c4d-0b5c2da27293 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:27.420549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15de1c-699c-48c3-8c4d-0b5c2da27293 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:27.420667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15de1c-699c-48c3-8c4d-0b5c2da27293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:27.429166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15de1c-699c-48c3-8c4d-0b5c2da27293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:27.431471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15de1c-699c-48c3-8c4d-0b5c2da27293 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:27.440679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1098/2217] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:27.526920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e2535-bb72-4c6e-bfad-cfc3063c0988 None None] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:27.527218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e2535-bb72-4c6e-bfad-cfc3063c0988 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:27.527502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e2535-bb72-4c6e-bfad-cfc3063c0988 None None] PATH_INFO: `/` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:27.528939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1120/2218] 10.222.0.40 () {54 vars in 921 bytes} [Mon Jul 25 06:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 25 06:02:27.534629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0760d-fb87-4bd9-8562-22ac32aac4ba None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:27.535034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0760d-fb87-4bd9-8562-22ac32aac4ba None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:27.535478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0760d-fb87-4bd9-8562-22ac32aac4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:27.554146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-fac0760d-fb87-4bd9-8562-22ac32aac4ba None None] MFA Rules not processed for user `e6a0f84bf3f245bdab1391a2451cb4c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:27.555356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fac0760d-fb87-4bd9-8562-22ac32aac4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:27.565520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1099/2219] 10.222.0.40 () {58 vars in 1018 bytes} [Mon Jul 25 06:02:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:02:27.577376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e2535-bb72-4c6e-bfad-cfc3063c0988 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:27.588993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e2535-bb72-4c6e-bfad-cfc3063c0988 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:27.592737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:27.593377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:27.593661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:27.593962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:27.605403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:27.606878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:27.621240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1121/2220] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:27.715777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3051ad6-716c-4fbb-b30a-d9a2a7a4c1b9 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:27.716070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3051ad6-716c-4fbb-b30a-d9a2a7a4c1b9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:27.716396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3051ad6-716c-4fbb-b30a-d9a2a7a4c1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:27.734443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e3051ad6-716c-4fbb-b30a-d9a2a7a4c1b9 None None] MFA Rules not processed for user `66f64211ceb949d398e47bc18e2abbca`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:27.743292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e3051ad6-716c-4fbb-b30a-d9a2a7a4c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:27.822007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1100/2221] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:31.831490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:31.832934 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:31.916813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a13048-a12f-429a-ad14-109c824e8cf8 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:31.922025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:31.922694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:31.922980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:31.923262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:31.925297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:31.938862 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:31.940535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:31.952131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1122/2222] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:02:36.242392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32a99-ba8a-4bb6-af4c-add18a51e333 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.242876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32a99-ba8a-4bb6-af4c-add18a51e333 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.242997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa32a99-ba8a-4bb6-af4c-add18a51e333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.262007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-5fa32a99-ba8a-4bb6-af4c-add18a51e333 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:36.263250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5fa32a99-ba8a-4bb6-af4c-add18a51e333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:36.270609 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1101/2223] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:02:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:36.282193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:36.293504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2a584-83c9-4d70-8d53-655813d1bbb6 service placement] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:36.298426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07a7fe-3afc-4d87-a9eb-9c89fff9280d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:36.299060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a7fe-3afc-4d87-a9eb-9c89fff9280d service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:36.299340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a7fe-3afc-4d87-a9eb-9c89fff9280d service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:36.299611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07a7fe-3afc-4d87-a9eb-9c89fff9280d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:36.308141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07a7fe-3afc-4d87-a9eb-9c89fff9280d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:36.309701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07a7fe-3afc-4d87-a9eb-9c89fff9280d service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:36.317764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1123/2224] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:02:37.187114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407cb8b-64cb-4d50-a2e9-2a40fe57bc5a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:37.187400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407cb8b-64cb-4d50-a2e9-2a40fe57bc5a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:37.187678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407cb8b-64cb-4d50-a2e9-2a40fe57bc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:37.205101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f407cb8b-64cb-4d50-a2e9-2a40fe57bc5a None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:37.206289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f407cb8b-64cb-4d50-a2e9-2a40fe57bc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:37.214163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1102/2225] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 25 06:02:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:37.226593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07a7fe-3afc-4d87-a9eb-9c89fff9280d service swift] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:37.237749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07a7fe-3afc-4d87-a9eb-9c89fff9280d service swift] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:37.242694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12d861-a543-475d-8362-518de2e83052 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:37.243362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d861-a543-475d-8362-518de2e83052 service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:37.243635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d861-a543-475d-8362-518de2e83052 service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:37.243911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d861-a543-475d-8362-518de2e83052 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:37.251815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12d861-a543-475d-8362-518de2e83052 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:37.253417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12d861-a543-475d-8362-518de2e83052 service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:37.261440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1124/2226] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:02:37.966239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952616e0-d2a8-4a4c-ba67-d3ea4a0232e3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:37.966739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952616e0-d2a8-4a4c-ba67-d3ea4a0232e3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:37.966826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952616e0-d2a8-4a4c-ba67-d3ea4a0232e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:37.993724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-952616e0-d2a8-4a4c-ba67-d3ea4a0232e3 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:37.995360 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-952616e0-d2a8-4a4c-ba67-d3ea4a0232e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:38.003035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1103/2227] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:02:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:38.018571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12d861-a543-475d-8362-518de2e83052 service swift] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:38.022914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:38.029637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12d861-a543-475d-8362-518de2e83052 service swift] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:38.036049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a7bd1-b5fa-41e6-bb7b-50bf76fede7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:38.039124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a7bd1-b5fa-41e6-bb7b-50bf76fede7e service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:38.039124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a7bd1-b5fa-41e6-bb7b-50bf76fede7e service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:38.039124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a7bd1-b5fa-41e6-bb7b-50bf76fede7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:38.048077 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a7bd1-b5fa-41e6-bb7b-50bf76fede7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:38.049548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a7bd1-b5fa-41e6-bb7b-50bf76fede7e service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:38.058759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1125/2228] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:02:38.524776 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3d9b-a570-4307-9c02-9c395c476cf0 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:38.525073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3d9b-a570-4307-9c02-9c395c476cf0 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:38.525359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3d9b-a570-4307-9c02-9c395c476cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:38.547104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c00c3d9b-a570-4307-9c02-9c395c476cf0 None None] MFA Rules not processed for user `5caf24262d214d9aa8cf9612582154f5`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:38.548547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c00c3d9b-a570-4307-9c02-9c395c476cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:38.622951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1104/2229] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:38.734716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31498a-c1e6-4557-bb17-ee898c30b70c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:38.735014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31498a-c1e6-4557-bb17-ee898c30b70c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:38.735300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d31498a-c1e6-4557-bb17-ee898c30b70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:38.768486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2d31498a-c1e6-4557-bb17-ee898c30b70c None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:38.770058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2d31498a-c1e6-4557-bb17-ee898c30b70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:38.775532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555911-1215-4836-a27c-f1023fcded93 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:38.775836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555911-1215-4836-a27c-f1023fcded93 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:38.776222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4555911-1215-4836-a27c-f1023fcded93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:38.780799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1126/2230] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:02:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:38.783121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:38.795433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31498a-c1e6-4557-bb17-ee898c30b70c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:38.796986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f4555911-1215-4836-a27c-f1023fcded93 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:38.798196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f4555911-1215-4836-a27c-f1023fcded93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:38.807053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1105/2231] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 25 06:02:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:02:38.812625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d31498a-c1e6-4557-bb17-ee898c30b70c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:38.820501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d6647-ec62-4163-915d-d4d87912bff2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:38.821515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6647-ec62-4163-915d-d4d87912bff2 service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:38.821793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6647-ec62-4163-915d-d4d87912bff2 service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:38.822046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4555911-1215-4836-a27c-f1023fcded93 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:38.822170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d6647-ec62-4163-915d-d4d87912bff2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:38.831777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d6647-ec62-4163-915d-d4d87912bff2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:38.833423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d6647-ec62-4163-915d-d4d87912bff2 service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:38.836615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4555911-1215-4836-a27c-f1023fcded93 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:38.842390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1127/2232] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:02:38.853804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a5bf6-996d-4c5a-a46c-740ed5f06b27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:38.854727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a5bf6-996d-4c5a-a46c-740ed5f06b27 service swift] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:38.855084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a5bf6-996d-4c5a-a46c-740ed5f06b27 service swift] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:38.855494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a5bf6-996d-4c5a-a46c-740ed5f06b27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:38.865018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a5bf6-996d-4c5a-a46c-740ed5f06b27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:38.866627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a5bf6-996d-4c5a-a46c-740ed5f06b27 service swift] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:38.876381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1106/2233] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 06:02:41.764350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d6647-ec62-4163-915d-d4d87912bff2 service swift] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:41.898325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d6647-ec62-4163-915d-d4d87912bff2 service swift] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:41.909590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:41.911058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:41.911543 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:41.912024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] PATH_INFO: `/v3/users/ff863f4f7a2147239fb491097268c9e8` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:41.913718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff863f4f7a2147239fb491097268c9e8)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:41.917241 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:42.114789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff863f4f7a2147239fb491097268c9e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.175063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff863f4f7a2147239fb491097268c9e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.175716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff863f4f7a2147239fb491097268c9e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.183841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff863f4f7a2147239fb491097268c9e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.184515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff863f4f7a2147239fb491097268c9e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.203029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1128/2234] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:02:41 2022] DELETE /identity/v3/users/ff863f4f7a2147239fb491097268c9e8 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:42.207649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abce3-1863-49a8-9fe1-46e8a61d7c72 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:42.209885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abce3-1863-49a8-9fe1-46e8a61d7c72 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:42.209885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3abce3-1863-49a8-9fe1-46e8a61d7c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:42.226657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-de3abce3-1863-49a8-9fe1-46e8a61d7c72 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:42.228635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-de3abce3-1863-49a8-9fe1-46e8a61d7c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:42.301488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1107/2235] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:02:42.312195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:42.314133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:42.382723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:42.388774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b276e-59c6-4c3c-af54-aff8f2bde8f3 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:42.392736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d402dd-96c8-4651-8cfb-b77c6b6cd40b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:42.398785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d402dd-96c8-4651-8cfb-b77c6b6cd40b service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:42.398785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d402dd-96c8-4651-8cfb-b77c6b6cd40b service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:42.398785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d402dd-96c8-4651-8cfb-b77c6b6cd40b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:42.404448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d402dd-96c8-4651-8cfb-b77c6b6cd40b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:42.405932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d402dd-96c8-4651-8cfb-b77c6b6cd40b service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:42.415394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1129/2236] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:02:42.676450 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3abce3-1863-49a8-9fe1-46e8a61d7c72 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:42.678057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-de3abce3-1863-49a8-9fe1-46e8a61d7c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:42.692090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3abce3-1863-49a8-9fe1-46e8a61d7c72 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:42.701000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:42.708259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:42.708259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:42.708259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] PATH_INFO: `/v3/projects/f0e6671f7e7e432b92deccdaf094fed4` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:42.708259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e6671f7e7e432b92deccdaf094fed4)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:42.708259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:42.849739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e6671f7e7e432b92deccdaf094fed4', 'request_id': 'req-42fe1016-dcb9-4142-a1c3-bdd897e7663c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.946984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e6671f7e7e432b92deccdaf094fed4', 'request_id': 'req-42fe1016-dcb9-4142-a1c3-bdd897e7663c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.953445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e6671f7e7e432b92deccdaf094fed4', 'request_id': 'req-42fe1016-dcb9-4142-a1c3-bdd897e7663c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.966780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] The token cache is being invalidate because project f0e6671f7e7e432b92deccdaf094fed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:02:42.967189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.967863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42fe1016-dcb9-4142-a1c3-bdd897e7663c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:02:42.969536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1108/2237] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:02:42 2022] DELETE /identity/v3/projects/f0e6671f7e7e432b92deccdaf094fed4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:42.975956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586a005-726b-4fc8-8f70-f240950bd569 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:42.976531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586a005-726b-4fc8-8f70-f240950bd569 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:42.976654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b586a005-726b-4fc8-8f70-f240950bd569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:42.999863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b586a005-726b-4fc8-8f70-f240950bd569 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:43.001739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b586a005-726b-4fc8-8f70-f240950bd569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:43.104286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1130/2238] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:02:43.111531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b40c3-ce13-442b-9ba5-a35d53ac53a9 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.111824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b40c3-ce13-442b-9ba5-a35d53ac53a9 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.112091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b40c3-ce13-442b-9ba5-a35d53ac53a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.131254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e63b40c3-ce13-442b-9ba5-a35d53ac53a9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:43.132648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e63b40c3-ce13-442b-9ba5-a35d53ac53a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:43.141405 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1109/2239] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:43.146097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586a005-726b-4fc8-8f70-f240950bd569 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:43.155886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b586a005-726b-4fc8-8f70-f240950bd569 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:43.160452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:43.161110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.161449 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.161676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.162407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:02:43.162719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:43.164365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:43.176279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1131/2240] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:02:43.180372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b40c3-ce13-442b-9ba5-a35d53ac53a9 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:43.190161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b40c3-ce13-442b-9ba5-a35d53ac53a9 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:43.194886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf4813-fe71-457e-8434-ebeec15bb363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:43.195536 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf4813-fe71-457e-8434-ebeec15bb363 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.195823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf4813-fe71-457e-8434-ebeec15bb363 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.196103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf4813-fe71-457e-8434-ebeec15bb363 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.197681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf4813-fe71-457e-8434-ebeec15bb363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:43.199606 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf4813-fe71-457e-8434-ebeec15bb363 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:43.281705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1110/2241] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:02:43 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:02:43.283041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:43.293861 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eda4f-4f5b-4223-8777-f63edbd9c012 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:43.348976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3384f0-9be2-439e-9a4a-46c926b2e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:43.349649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3384f0-9be2-439e-9a4a-46c926b2e058 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.349936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3384f0-9be2-439e-9a4a-46c926b2e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.350216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3384f0-9be2-439e-9a4a-46c926b2e058 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.350904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3384f0-9be2-439e-9a4a-46c926b2e058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:43.353464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3384f0-9be2-439e-9a4a-46c926b2e058 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:43.434326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1132/2242] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:02:43 2022] POST /identity/v3/users => generated 292 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:02:43.438215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf4813-fe71-457e-8434-ebeec15bb363 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:43.457690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf4813-fe71-457e-8434-ebeec15bb363 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:43.462719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e450-8d3d-438f-9e6e-868466361576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:43.464604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e450-8d3d-438f-9e6e-868466361576 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.464604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e450-8d3d-438f-9e6e-868466361576 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.466448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e450-8d3d-438f-9e6e-868466361576 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.467303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4e450-8d3d-438f-9e6e-868466361576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:43.471350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4e450-8d3d-438f-9e6e-868466361576 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:43.490031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1111/2243] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:02:43.494685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3384f0-9be2-439e-9a4a-46c926b2e058 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:43.505312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3384f0-9be2-439e-9a4a-46c926b2e058 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:43.510090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdbfff-2e32-46f8-8971-4284dfba2c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:43.510782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdbfff-2e32-46f8-8971-4284dfba2c55 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.511098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdbfff-2e32-46f8-8971-4284dfba2c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.511339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdbfff-2e32-46f8-8971-4284dfba2c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.512081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdbfff-2e32-46f8-8971-4284dfba2c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:43.513748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdbfff-2e32-46f8-8971-4284dfba2c55 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:43.525804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1133/2244] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:02:43.530181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e450-8d3d-438f-9e6e-868466361576 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:43.541304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e450-8d3d-438f-9e6e-868466361576 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:43.546113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:43.547023 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.547310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.547610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] PATH_INFO: `/v3/projects/a18a468edefd401c835e7e368ac740e5/users/e13b80595f1d4c8cb4522657c9a88cd6/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.565404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18a468edefd401c835e7e368ac740e5, user_id=e13b80595f1d4c8cb4522657c9a88cd6, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:43.568067 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:43.637468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1112/2245] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:02:43 2022] PUT /identity/v3/projects/a18a468edefd401c835e7e368ac740e5/users/e13b80595f1d4c8cb4522657c9a88cd6/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:43.642539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53a663-eb13-4829-b150-92e076348747 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.642886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53a663-eb13-4829-b150-92e076348747 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.643197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f53a663-eb13-4829-b150-92e076348747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.663274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4f53a663-eb13-4829-b150-92e076348747 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:43.664562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4f53a663-eb13-4829-b150-92e076348747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:43.719878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:43.730095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1134/2246] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:43.740389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:43.742306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:43.749167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:43.820735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21717b-1aa6-4460-9781-b29121556b19 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:43.825705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c24cfe-ce84-4c9e-8dd3-768d1f417097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:43.826323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24cfe-ce84-4c9e-8dd3-768d1f417097 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:43.826603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24cfe-ce84-4c9e-8dd3-768d1f417097 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:43.826912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24cfe-ce84-4c9e-8dd3-768d1f417097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:43.838018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c24cfe-ce84-4c9e-8dd3-768d1f417097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:43.842018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c24cfe-ce84-4c9e-8dd3-768d1f417097 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:43.849481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1113/2247] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:50.765308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57d84b-a156-483a-b248-1158100efe87 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:50.765308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57d84b-a156-483a-b248-1158100efe87 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:50.765308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57d84b-a156-483a-b248-1158100efe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:50.833714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-3d57d84b-a156-483a-b248-1158100efe87 None None] MFA Rules not processed for user `e13b80595f1d4c8cb4522657c9a88cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:50.835400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3d57d84b-a156-483a-b248-1158100efe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:50.932715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1135/2248] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:50.937409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c24cfe-ce84-4c9e-8dd3-768d1f417097 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:50.948392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c24cfe-ce84-4c9e-8dd3-768d1f417097 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:50.954151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc4a29-0ba6-4e80-b1a3-862d56c8034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:50.954820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc4a29-0ba6-4e80-b1a3-862d56c8034b admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:50.955106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc4a29-0ba6-4e80-b1a3-862d56c8034b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:50.958981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc4a29-0ba6-4e80-b1a3-862d56c8034b admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:50.959905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc4a29-0ba6-4e80-b1a3-862d56c8034b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:50.961955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc4a29-0ba6-4e80-b1a3-862d56c8034b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:51.222837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1114/2249] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:02:50 2022] POST /identity/v3/projects => generated 383 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:02:51.226946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57d84b-a156-483a-b248-1158100efe87 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:51.237822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57d84b-a156-483a-b248-1158100efe87 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:51.304617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3359a-c3d5-419e-83da-8e4341cf50dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:51.305327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3359a-c3d5-419e-83da-8e4341cf50dd admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:51.305588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3359a-c3d5-419e-83da-8e4341cf50dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:51.305873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3359a-c3d5-419e-83da-8e4341cf50dd admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:51.307360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3359a-c3d5-419e-83da-8e4341cf50dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:51.309352 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d3359a-c3d5-419e-83da-8e4341cf50dd admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:51.382263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1136/2250] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:02:51 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:02:51.387221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc4a29-0ba6-4e80-b1a3-862d56c8034b admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:51.398892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc4a29-0ba6-4e80-b1a3-862d56c8034b admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:51.404398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677e64-4a8f-4834-b215-ed0ac2fb695d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:51.405169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677e64-4a8f-4834-b215-ed0ac2fb695d admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:51.405530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677e64-4a8f-4834-b215-ed0ac2fb695d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:51.405887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677e64-4a8f-4834-b215-ed0ac2fb695d admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:51.406778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e677e64-4a8f-4834-b215-ed0ac2fb695d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:51.408488 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e677e64-4a8f-4834-b215-ed0ac2fb695d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:51.420295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1115/2251] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:02:51.424145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3359a-c3d5-419e-83da-8e4341cf50dd admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:51.435769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3359a-c3d5-419e-83da-8e4341cf50dd admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:51.441590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52daba-c191-4df6-909c-281ee4b585b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:51.441801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52daba-c191-4df6-909c-281ee4b585b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:51.442096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52daba-c191-4df6-909c-281ee4b585b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:51.442379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c52daba-c191-4df6-909c-281ee4b585b6 admin admin] PATH_INFO: `/v3/projects/2c590ee21b9f407aba4b79240f33394a/users/9a8f3d909c464a08bc398ec9cbd64e3e/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:51.459032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52daba-c191-4df6-909c-281ee4b585b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c590ee21b9f407aba4b79240f33394a, user_id=9a8f3d909c464a08bc398ec9cbd64e3e, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:51.461784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c52daba-c191-4df6-909c-281ee4b585b6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:51.579557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1137/2252] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:02:51 2022] PUT /identity/v3/projects/2c590ee21b9f407aba4b79240f33394a/users/9a8f3d909c464a08bc398ec9cbd64e3e/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:02:51.583447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677e64-4a8f-4834-b215-ed0ac2fb695d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:51.593779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677e64-4a8f-4834-b215-ed0ac2fb695d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:51.652810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ffc78-e5f5-4975-9104-510185da261e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:51.653487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ffc78-e5f5-4975-9104-510185da261e admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:51.653772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ffc78-e5f5-4975-9104-510185da261e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:51.654051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ffc78-e5f5-4975-9104-510185da261e admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:51.654785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ffc78-e5f5-4975-9104-510185da261e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:51.656402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ffc78-e5f5-4975-9104-510185da261e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:51.671793 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1116/2253] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:02:51.673237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52daba-c191-4df6-909c-281ee4b585b6 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:51.686013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c52daba-c191-4df6-909c-281ee4b585b6 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:51.690788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:51.691829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:51.692119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:51.692439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] PATH_INFO: `/v3/projects/2c590ee21b9f407aba4b79240f33394a/users/9a8f3d909c464a08bc398ec9cbd64e3e/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:51.694466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c590ee21b9f407aba4b79240f33394a, user_id=9a8f3d909c464a08bc398ec9cbd64e3e, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:51.697489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:51.708175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1138/2254] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:02:51 2022] PUT /identity/v3/projects/2c590ee21b9f407aba4b79240f33394a/users/9a8f3d909c464a08bc398ec9cbd64e3e/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:02:58.551027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eabb1-4e2e-43cb-a673-593b674a6bf1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:58.551585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eabb1-4e2e-43cb-a673-593b674a6bf1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:58.551783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eabb1-4e2e-43cb-a673-593b674a6bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:58.582188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-343eabb1-4e2e-43cb-a673-593b674a6bf1 None None] MFA Rules not processed for user `9a8f3d909c464a08bc398ec9cbd64e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:58.584007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-343eabb1-4e2e-43cb-a673-593b674a6bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:58.654250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:58.668467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1117/2255] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:02:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:58.682539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:58.684321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:58.749459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:02:58.765020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2872-c372-4054-b1b4-7fbd52060e82 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:58.771495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c0883bc26a04de2916c78df342d2023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:58.772186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:58.772473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:58.772757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:58.839775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:58.841329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:58.853885 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1139/2256] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:02:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:02:59.118910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467655e4-ff62-44db-8787-dd6997050346 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:59.119201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467655e4-ff62-44db-8787-dd6997050346 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:59.119487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467655e4-ff62-44db-8787-dd6997050346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:59.138236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-467655e4-ff62-44db-8787-dd6997050346 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:02:59.139455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-467655e4-ff62-44db-8787-dd6997050346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:59.204589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1118/2257] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:02:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:02:59.218507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:02:59.219784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:02:59.288457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bce2ef-21c8-4815-b286-7ae69c5c2a84 service glance] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:02:59.293428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f356ad2-e8f4-4019-80c1-67011e6d0915 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:02:59.294095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f356ad2-e8f4-4019-80c1-67011e6d0915 service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:02:59.294378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f356ad2-e8f4-4019-80c1-67011e6d0915 service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:02:59.294655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f356ad2-e8f4-4019-80c1-67011e6d0915 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:02:59.302670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f356ad2-e8f4-4019-80c1-67011e6d0915 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:02:59.304293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f356ad2-e8f4-4019-80c1-67011e6d0915 service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:02:59.312426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1140/2258] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:02:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:09.197919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=28 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:09.309178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=62 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:11.405787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38624e43-0cef-42f1-9e27-eb2013445439 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:11.406301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38624e43-0cef-42f1-9e27-eb2013445439 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:11.406396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38624e43-0cef-42f1-9e27-eb2013445439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:11.429660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-38624e43-0cef-42f1-9e27-eb2013445439 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:11.431035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-38624e43-0cef-42f1-9e27-eb2013445439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:11.439313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1119/2259] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:03:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:11.456431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f356ad2-e8f4-4019-80c1-67011e6d0915 service swift] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:11.467409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f356ad2-e8f4-4019-80c1-67011e6d0915 service swift] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:11.473507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c431-62ed-4007-9dc9-9ea3907b424c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:11.474403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c431-62ed-4007-9dc9-9ea3907b424c service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:11.474762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c431-62ed-4007-9dc9-9ea3907b424c service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:11.475133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca5c431-62ed-4007-9dc9-9ea3907b424c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:11.484793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5c431-62ed-4007-9dc9-9ea3907b424c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:11.486617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca5c431-62ed-4007-9dc9-9ea3907b424c service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:11.496825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1141/2260] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:03:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:14.723718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59166d-f4e2-4d62-a8b3-6236c18e9a95 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:14.724944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59166d-f4e2-4d62-a8b3-6236c18e9a95 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:14.725230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59166d-f4e2-4d62-a8b3-6236c18e9a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:14.753894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-1e59166d-f4e2-4d62-a8b3-6236c18e9a95 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:14.755715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1e59166d-f4e2-4d62-a8b3-6236c18e9a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:14.764571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1120/2261] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 25 06:03:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:14.781829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c431-62ed-4007-9dc9-9ea3907b424c service swift] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:14.795256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca5c431-62ed-4007-9dc9-9ea3907b424c service swift] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:14.814906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616e176-48f6-428b-af06-bbcdd1059031 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:14.815608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616e176-48f6-428b-af06-bbcdd1059031 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:14.816104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616e176-48f6-428b-af06-bbcdd1059031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:14.817534 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b75ca87-c069-4890-aa52-452b579879a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:14.818204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75ca87-c069-4890-aa52-452b579879a2 service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:14.818488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75ca87-c069-4890-aa52-452b579879a2 service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:14.818766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b75ca87-c069-4890-aa52-452b579879a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:14.834774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75ca87-c069-4890-aa52-452b579879a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:14.836776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b75ca87-c069-4890-aa52-452b579879a2 service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:14.841168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9616e176-48f6-428b-af06-bbcdd1059031 None None] MFA Rules not processed for user `e13b80595f1d4c8cb4522657c9a88cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:14.843929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9616e176-48f6-428b-af06-bbcdd1059031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:14.848422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1142/2262] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:03:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:14.859864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1121/2263] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:03:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:03:21.480093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10d897-b3ed-49c6-9bf1-643bb2afdc84 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:21.480676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10d897-b3ed-49c6-9bf1-643bb2afdc84 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:21.480774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10d897-b3ed-49c6-9bf1-643bb2afdc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:21.505696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-5b10d897-b3ed-49c6-9bf1-643bb2afdc84 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:21.507512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5b10d897-b3ed-49c6-9bf1-643bb2afdc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:21.525988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1143/2264] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:03:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:21.548872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616e176-48f6-428b-af06-bbcdd1059031 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:21.572622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616e176-48f6-428b-af06-bbcdd1059031 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:21.578954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959fa85-6a0f-458a-a94f-e68d883bc659 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:21.580068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959fa85-6a0f-458a-a94f-e68d883bc659 service swift] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:21.580632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959fa85-6a0f-458a-a94f-e68d883bc659 service swift] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:21.582603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959fa85-6a0f-458a-a94f-e68d883bc659 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:21.593055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959fa85-6a0f-458a-a94f-e68d883bc659 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:21.594890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959fa85-6a0f-458a-a94f-e68d883bc659 service swift] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:21.607907 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1122/2265] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:03:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:25.975858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5cd5-5a51-4734-879e-242b752398dd None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:25.976347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5cd5-5a51-4734-879e-242b752398dd None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:25.976433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b5cd5-5a51-4734-879e-242b752398dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:26.044151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-154b5cd5-5a51-4734-879e-242b752398dd None None] MFA Rules not processed for user `8eedefd8d94047478663ceab8962d738`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:26.046066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-154b5cd5-5a51-4734-879e-242b752398dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:26.403552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1144/2266] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:03:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:31.602115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=15 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:36.017922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4176d8-49a1-4de7-b8ea-377b833b7a17 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:36.018543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4176d8-49a1-4de7-b8ea-377b833b7a17 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:36.018543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4176d8-49a1-4de7-b8ea-377b833b7a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:36.038038 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ee4176d8-49a1-4de7-b8ea-377b833b7a17 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:36.039885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ee4176d8-49a1-4de7-b8ea-377b833b7a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:36.047624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1123/2267] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 25 06:03:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:36.060503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5cd5-5a51-4734-879e-242b752398dd None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:36.073229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b5cd5-5a51-4734-879e-242b752398dd None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:36.080023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8ba59-c151-4a56-9241-f9595f0207a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:36.080876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ba59-c151-4a56-9241-f9595f0207a8 service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:36.081267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ba59-c151-4a56-9241-f9595f0207a8 service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:36.081661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8ba59-c151-4a56-9241-f9595f0207a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:36.091239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8ba59-c151-4a56-9241-f9595f0207a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:36.092980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8ba59-c151-4a56-9241-f9595f0207a8 service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:36.102107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1145/2268] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:03:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:03:39.250947 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4176d8-49a1-4de7-b8ea-377b833b7a17 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:39.252546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ee4176d8-49a1-4de7-b8ea-377b833b7a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:39.334698 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4176d8-49a1-4de7-b8ea-377b833b7a17 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:39.339912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:39.340824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:39.341112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:39.341392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] PATH_INFO: `/v3/users/66f64211ceb949d398e47bc18e2abbca` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:39.342600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f64211ceb949d398e47bc18e2abbca)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:39.344571 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:39.565548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f64211ceb949d398e47bc18e2abbca'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.662635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f64211ceb949d398e47bc18e2abbca'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.663416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f64211ceb949d398e47bc18e2abbca'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.671924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f64211ceb949d398e47bc18e2abbca'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.672783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c7b301fc-a7af-48f3-bd78-5e37c8b3e3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f64211ceb949d398e47bc18e2abbca'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:39.698053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1124/2269] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:03:39 2022] DELETE /identity/v3/users/66f64211ceb949d398e47bc18e2abbca => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:03:40.006911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8ba59-c151-4a56-9241-f9595f0207a8 service swift] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:40.013170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=47 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:40.019316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8ba59-c151-4a56-9241-f9595f0207a8 service swift] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:03:40.078915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:03:40.080340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:40.080340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:40.080340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] PATH_INFO: `/v3/projects/e3f62f9df69f47d0ad40154526b95be1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:40.081510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3f62f9df69f47d0ad40154526b95be1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:03:40.083412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:03:40.205442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3f62f9df69f47d0ad40154526b95be1', 'request_id': 'req-ab3d3122-54dd-4973-af67-169a464d73c9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:40.279710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3f62f9df69f47d0ad40154526b95be1', 'request_id': 'req-ab3d3122-54dd-4973-af67-169a464d73c9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:40.286509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3f62f9df69f47d0ad40154526b95be1', 'request_id': 'req-ab3d3122-54dd-4973-af67-169a464d73c9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:40.299600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] The token cache is being invalidate because project e3f62f9df69f47d0ad40154526b95be1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:03:40.302343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:40.302343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:03:40.302343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1146/2270] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:03:40 2022] DELETE /identity/v3/projects/e3f62f9df69f47d0ad40154526b95be1 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:03:43.383158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f81938-4b19-4fa6-8136-f723ce4add6a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:03:43.388434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f81938-4b19-4fa6-8136-f723ce4add6a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:03:43.388434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f81938-4b19-4fa6-8136-f723ce4add6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:03:43.401903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c7f81938-4b19-4fa6-8136-f723ce4add6a None None] MFA Rules not processed for user `62e566aa74b045cb8cc6e8edf4a256e1`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:03:43.403144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c7f81938-4b19-4fa6-8136-f723ce4add6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:03:43.458706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1125/2271] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:03:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:03:50.296506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:53.451867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:03:59.937351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:03:59.938838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:00.010119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d3122-54dd-4973-af67-169a464d73c9 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:00.014668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:00.015366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:00.015653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:00.015940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:00.018020 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:00.096580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:00.098051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:00.111400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1147/2272] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:03:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:10.105722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=50 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:17.148341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b057-57b7-4fed-b71d-c8b30e7141b8 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:17.148877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b057-57b7-4fed-b71d-c8b30e7141b8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:17.148976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010b057-57b7-4fed-b71d-c8b30e7141b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:17.166929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-8010b057-57b7-4fed-b71d-c8b30e7141b8 None None] MFA Rules not processed for user `d485f9fc38a442dd9e1528024460fcca`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:17.168250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8010b057-57b7-4fed-b71d-c8b30e7141b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:17.226925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1126/2273] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:04:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:04:18.333876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:18.333876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:18.412464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ba7d8-f940-493a-9de7-2da49f8269b0 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:18.417651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:18.418525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:18.418806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:18.419081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] PATH_INFO: `/v3/users/5caf24262d214d9aa8cf9612582154f5` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:18.420395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5caf24262d214d9aa8cf9612582154f5)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:18.422990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:18.634424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5caf24262d214d9aa8cf9612582154f5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:18.712784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5caf24262d214d9aa8cf9612582154f5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:18.713464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5caf24262d214d9aa8cf9612582154f5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:18.721699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5caf24262d214d9aa8cf9612582154f5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:18.722341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0b27cb19-a175-4aa7-9843-85aa7ae13a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5caf24262d214d9aa8cf9612582154f5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:18.739632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1148/2274] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:04:18 2022] DELETE /identity/v3/users/5caf24262d214d9aa8cf9612582154f5 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:04:18.997276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010b057-57b7-4fed-b71d-c8b30e7141b8 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:19.012383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010b057-57b7-4fed-b71d-c8b30e7141b8 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:19.088501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:19.089375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.089659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.089938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] PATH_INFO: `/v3/projects/7d97f354896a4f1bbef0b4d1e6b99c27` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.093575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d97f354896a4f1bbef0b4d1e6b99c27)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:19.093575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:19.235139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d97f354896a4f1bbef0b4d1e6b99c27', 'request_id': 'req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:19.315300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d97f354896a4f1bbef0b4d1e6b99c27', 'request_id': 'req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:19.321554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d97f354896a4f1bbef0b4d1e6b99c27', 'request_id': 'req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:19.332921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] The token cache is being invalidate because project 7d97f354896a4f1bbef0b4d1e6b99c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:04:19.333210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:19.333867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ebb328cf-73bf-4f39-b9af-23e9a70f8dbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:19.335400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1127/2275] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:04:18 2022] DELETE /identity/v3/projects/7d97f354896a4f1bbef0b4d1e6b99c27 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:04:19.340469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75da15-ecaa-414d-8621-f297f1d1edbd None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.340994 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75da15-ecaa-414d-8621-f297f1d1edbd None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.341105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75da15-ecaa-414d-8621-f297f1d1edbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.361920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4f75da15-ecaa-414d-8621-f297f1d1edbd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:19.363066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4f75da15-ecaa-414d-8621-f297f1d1edbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:19.427261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1149/2276] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:04:19.435242 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b454-02e5-459f-9e8e-9f4e7e964250 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.435730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b454-02e5-459f-9e8e-9f4e7e964250 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.436443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30b454-02e5-459f-9e8e-9f4e7e964250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.461641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7d30b454-02e5-459f-9e8e-9f4e7e964250 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:19.465208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7d30b454-02e5-459f-9e8e-9f4e7e964250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:19.472101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1128/2277] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:04:19.482433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75da15-ecaa-414d-8621-f297f1d1edbd None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:19.495052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75da15-ecaa-414d-8621-f297f1d1edbd None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:19.500828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:19.501484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.501769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.502046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.502779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:04:19.503095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:19.504756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:19.516936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1150/2278] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:04:19.521433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30b454-02e5-459f-9e8e-9f4e7e964250 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:19.532084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30b454-02e5-459f-9e8e-9f4e7e964250 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:19.537467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85059c3-62ca-4f89-b462-b90123f6069f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:19.538143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85059c3-62ca-4f89-b462-b90123f6069f admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.538663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85059c3-62ca-4f89-b462-b90123f6069f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.538855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85059c3-62ca-4f89-b462-b90123f6069f admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.539604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85059c3-62ca-4f89-b462-b90123f6069f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:19.542319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85059c3-62ca-4f89-b462-b90123f6069f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:19.630241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1129/2279] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:04:19 2022] POST /identity/v3/projects => generated 397 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:04:19.634272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:19.645432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf85952f-856d-4890-b5f0-8d997647ea8f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:19.649755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:19.702296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9a2d6-e468-4cbf-bc9a-9172115b2e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:19.703639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a2d6-e468-4cbf-bc9a-9172115b2e54 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.704247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a2d6-e468-4cbf-bc9a-9172115b2e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.704656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9a2d6-e468-4cbf-bc9a-9172115b2e54 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.705430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9a2d6-e468-4cbf-bc9a-9172115b2e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:19.707535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9a2d6-e468-4cbf-bc9a-9172115b2e54 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:19.799672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1151/2280] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:04:19 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:04:19.803441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85059c3-62ca-4f89-b462-b90123f6069f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:19.826112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85059c3-62ca-4f89-b462-b90123f6069f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:19.832449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ac76-452f-4d16-81ca-658b67b6b7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:19.833128 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ac76-452f-4d16-81ca-658b67b6b7da admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.833414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ac76-452f-4d16-81ca-658b67b6b7da admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.833808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708ac76-452f-4d16-81ca-658b67b6b7da admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.834564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708ac76-452f-4d16-81ca-658b67b6b7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:19.836467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708ac76-452f-4d16-81ca-658b67b6b7da admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:19.837992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:19.847934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1130/2281] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:04:19.852233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9a2d6-e468-4cbf-bc9a-9172115b2e54 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:19.862388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9a2d6-e468-4cbf-bc9a-9172115b2e54 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:19.867084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb88b-6763-4d16-a318-533eeafa64ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:19.867745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb88b-6763-4d16-a318-533eeafa64ec admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.868035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb88b-6763-4d16-a318-533eeafa64ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.878721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb88b-6763-4d16-a318-533eeafa64ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.878721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacb88b-6763-4d16-a318-533eeafa64ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:19.878721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacb88b-6763-4d16-a318-533eeafa64ec admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:19.881467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1152/2282] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:04:19.885648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ac76-452f-4d16-81ca-658b67b6b7da admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:19.896892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708ac76-452f-4d16-81ca-658b67b6b7da admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:19.902139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:19.905946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:19.905946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:19.905946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] PATH_INFO: `/v3/projects/6222c932c5674507b41a92d1bc41a2f6/users/cda1c1989f0e4098aa0a0dc5fd341bed/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:19.918670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6222c932c5674507b41a92d1bc41a2f6, user_id=cda1c1989f0e4098aa0a0dc5fd341bed, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:19.921381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:20.001952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1131/2283] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:04:19 2022] PUT /identity/v3/projects/6222c932c5674507b41a92d1bc41a2f6/users/cda1c1989f0e4098aa0a0dc5fd341bed/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:04:20.017646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a443-a237-4822-9748-d379766f3f6a None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:20.017937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a443-a237-4822-9748-d379766f3f6a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:20.018219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a443-a237-4822-9748-d379766f3f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:20.047242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d117a443-a237-4822-9748-d379766f3f6a None None] MFA Rules not processed for user `cda1c1989f0e4098aa0a0dc5fd341bed`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:20.048515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d117a443-a237-4822-9748-d379766f3f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:20.114086 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1153/2284] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:04:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:04:20.126223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:20.141374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:20.189285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32162034-0eda-47cf-a305-36172ce01851 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:20.192984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:20.193653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:20.193934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:20.194209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:20.204056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:20.205620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:20.215015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1132/2285] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:04:22.741267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117a443-a237-4822-9748-d379766f3f6a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:22.743259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d117a443-a237-4822-9748-d379766f3f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:22.835929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117a443-a237-4822-9748-d379766f3f6a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:22.840409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:22.841121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:22.841413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:22.841692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:22.844245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:22.892867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:22.945705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:22.947173 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:22.961120 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1154/2286] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:24.094108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:24.095731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:24.108995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fdc246-146b-40b8-be7d-5fe20018567a service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:24.112765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b12ec-d7a3-4f22-82b5-c44b9d0ed1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:24.113393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b12ec-d7a3-4f22-82b5-c44b9d0ed1ba service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:24.113678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b12ec-d7a3-4f22-82b5-c44b9d0ed1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:24.113958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b12ec-d7a3-4f22-82b5-c44b9d0ed1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:24.115704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f71b12ec-d7a3-4f22-82b5-c44b9d0ed1ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:24.130430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b12ec-d7a3-4f22-82b5-c44b9d0ed1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:24.131830 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b12ec-d7a3-4f22-82b5-c44b9d0ed1ba service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:24.143864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1133/2287] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:29.165762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:29.230860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf17ac7-6d6a-4296-ba4f-1a76a01c033f service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:29.235738 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:29.236641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:29.236924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:29.237194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] PATH_INFO: `/v3/users/cda1c1989f0e4098aa0a0dc5fd341bed` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:29.238387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda1c1989f0e4098aa0a0dc5fd341bed)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:29.240325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:29.472215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda1c1989f0e4098aa0a0dc5fd341bed'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:29.567150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda1c1989f0e4098aa0a0dc5fd341bed'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:29.568159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda1c1989f0e4098aa0a0dc5fd341bed'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:29.577238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda1c1989f0e4098aa0a0dc5fd341bed'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:29.578042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda1c1989f0e4098aa0a0dc5fd341bed'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:29.600828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1155/2288] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:04:29 2022] DELETE /identity/v3/users/cda1c1989f0e4098aa0a0dc5fd341bed => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:04:29.605387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe7d81-b151-4609-a305-d009d4179b14 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:29.606141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe7d81-b151-4609-a305-d009d4179b14 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:29.606141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe7d81-b151-4609-a305-d009d4179b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:29.625788 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-81fe7d81-b151-4609-a305-d009d4179b14 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:29.627143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-81fe7d81-b151-4609-a305-d009d4179b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:29.849322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1134/2289] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:04:29.860172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:29.861620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:29.902102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:29.919072 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7cfb4-ceec-4bef-8880-c8d46ef6ac54 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:29.923381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:29.924032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:29.924350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:29.924626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:29.933995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:29.935429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:29.946044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1156/2290] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:30.140435 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cffa5b-06fc-4b36-a1b2-934fe1688e7c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:30.144796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cffa5b-06fc-4b36-a1b2-934fe1688e7c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:30.144796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cffa5b-06fc-4b36-a1b2-934fe1688e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:30.163630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c5cffa5b-06fc-4b36-a1b2-934fe1688e7c None None] MFA Rules not processed for user `a7224418acee432182bf3195bfc8fd99`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:04:30.165316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c5cffa5b-06fc-4b36-a1b2-934fe1688e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:30.249091 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1135/2291] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:04:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:04:30.386843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:30.386843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:30.401499 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ee4a4-f23c-48f1-92b6-2a84821a5afd service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:30.412075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e17268-a450-449f-be72-59805500b649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:30.416424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e17268-a450-449f-be72-59805500b649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:30.416715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e17268-a450-449f-be72-59805500b649 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:30.417023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e17268-a450-449f-be72-59805500b649 admin admin] PATH_INFO: `/v3/projects/6222c932c5674507b41a92d1bc41a2f6` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:30.418354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e17268-a450-449f-be72-59805500b649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6222c932c5674507b41a92d1bc41a2f6)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:30.420437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e17268-a450-449f-be72-59805500b649 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:30.561508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56e17268-a450-449f-be72-59805500b649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6222c932c5674507b41a92d1bc41a2f6', 'request_id': 'req-56e17268-a450-449f-be72-59805500b649'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:30.647911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56e17268-a450-449f-be72-59805500b649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6222c932c5674507b41a92d1bc41a2f6', 'request_id': 'req-56e17268-a450-449f-be72-59805500b649'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:30.656311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56e17268-a450-449f-be72-59805500b649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6222c932c5674507b41a92d1bc41a2f6', 'request_id': 'req-56e17268-a450-449f-be72-59805500b649'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:30.671079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56e17268-a450-449f-be72-59805500b649 admin admin] The token cache is being invalidate because project 6222c932c5674507b41a92d1bc41a2f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:04:30.671372 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56e17268-a450-449f-be72-59805500b649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:30.672628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-56e17268-a450-449f-be72-59805500b649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:04:30.674321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1157/2292] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:04:30 2022] DELETE /identity/v3/projects/6222c932c5674507b41a92d1bc41a2f6 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:04:34.570787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cffa5b-06fc-4b36-a1b2-934fe1688e7c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:34.572363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c5cffa5b-06fc-4b36-a1b2-934fe1688e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:34.582373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:34.646592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cffa5b-06fc-4b36-a1b2-934fe1688e7c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:34.696759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e17268-a450-449f-be72-59805500b649 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:34.698230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-56e17268-a450-449f-be72-59805500b649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:34.711386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:34.712036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:34.712421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:34.712704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:34.714557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:34.758722 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e17268-a450-449f-be72-59805500b649 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:34.762513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:34.763167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:34.763454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:34.763731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:34.765567 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:34.786469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:34.788070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:34.798739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1136/2293] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:34.830954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:34.832599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:34.842110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1158/2294] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:04:40.779146 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:40.780924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:40.795589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13126a06-33d1-4b23-adf1-3d3a879720f5 service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:40.801894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:40.802558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:40.802901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:40.803125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:40.804922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:40.849419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:40.866837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:40.869850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:40.879575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1137/2295] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:44.840752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:45.948224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:04:45.949695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:45.965660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ab834-0564-43d3-b2a4-d999924a6bbd service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:04:45.971155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:04:45.972458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:04:45.973046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:04:45.973237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:04:45.974949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:04:46.042870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:04:46.044728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:04:46.056253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1159/2296] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:04:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:04:50.887202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=9 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:04:56.051483 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=28 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:05.243012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:05.249781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:05.256661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446cce8-bf3b-4611-881d-199e94689706 service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:05.260272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:05.260942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:05.261256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:05.261502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:05.263233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:05.334443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:05.335924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:05.345923 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1138/2297] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:05.479350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:05.492577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee601f-f550-462e-8a1c-b41f30c013df service nova] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:05.498671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:05.499673 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:05.500334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:05.500770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:05.502940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:05.518704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:05.521098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:05.534172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1160/2298] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:09.177938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:09.178429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:09.191590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6de547-a3d2-418b-8584-1ad7be14c20e service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:09.196449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:09.197384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:09.197665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:09.197945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] PATH_INFO: `/v3/users/62e566aa74b045cb8cc6e8edf4a256e1` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:09.199134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e566aa74b045cb8cc6e8edf4a256e1)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:09.201086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:09.393681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e566aa74b045cb8cc6e8edf4a256e1'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:09.459380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e566aa74b045cb8cc6e8edf4a256e1'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:09.459999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e566aa74b045cb8cc6e8edf4a256e1'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:09.467366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e566aa74b045cb8cc6e8edf4a256e1'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:09.468002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c1e004d6-7018-4b63-a3b4-fba074abd0ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e566aa74b045cb8cc6e8edf4a256e1'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:09.486805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1139/2299] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:09 2022] DELETE /identity/v3/users/62e566aa74b045cb8cc6e8edf4a256e1 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:09.747203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:09.758953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048b0e7-9089-495d-9137-c99b66eae4d2 service nova] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:09.828308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:09.829183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:09.829467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:09.829741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] PATH_INFO: `/v3/projects/07562543fa3e40d584b4d4525b112319` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:09.830921 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07562543fa3e40d584b4d4525b112319)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:09.833091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:09.957395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07562543fa3e40d584b4d4525b112319', 'request_id': 'req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:10.048049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07562543fa3e40d584b4d4525b112319', 'request_id': 'req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:10.054926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07562543fa3e40d584b4d4525b112319', 'request_id': 'req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:10.066569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] The token cache is being invalidate because project 07562543fa3e40d584b4d4525b112319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:10.066821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:10.067471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-36c99362-33cd-4cfb-873d-cd8e24fe8fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:10.069102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1161/2300] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:09 2022] DELETE /identity/v3/projects/07562543fa3e40d584b4d4525b112319 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:10.074136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77caf01-130b-4460-b28b-a3ac830ecf21 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.074427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77caf01-130b-4460-b28b-a3ac830ecf21 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.074725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77caf01-130b-4460-b28b-a3ac830ecf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.094223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f77caf01-130b-4460-b28b-a3ac830ecf21 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:10.099775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f77caf01-130b-4460-b28b-a3ac830ecf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:10.157014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1140/2301] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:10.165405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacd086-47b1-4b2d-a648-cdecf25674fe None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.165794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacd086-47b1-4b2d-a648-cdecf25674fe None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.166187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddacd086-47b1-4b2d-a648-cdecf25674fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.185162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ddacd086-47b1-4b2d-a648-cdecf25674fe None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:10.186467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ddacd086-47b1-4b2d-a648-cdecf25674fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:10.195127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1162/2302] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:10.199403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77caf01-130b-4460-b28b-a3ac830ecf21 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:10.209688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77caf01-130b-4460-b28b-a3ac830ecf21 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:10.214285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:10.214943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.215222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.215494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.216249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:10.216555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:10.218134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:10.219752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:10.231312 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1141/2303] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:05:10.235565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacd086-47b1-4b2d-a648-cdecf25674fe None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:10.245166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddacd086-47b1-4b2d-a648-cdecf25674fe None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:10.249855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9be2d5-5229-4a42-9e28-bf29ede9600e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:10.250502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be2d5-5229-4a42-9e28-bf29ede9600e admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.250784 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be2d5-5229-4a42-9e28-bf29ede9600e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.251052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be2d5-5229-4a42-9e28-bf29ede9600e admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.251745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9be2d5-5229-4a42-9e28-bf29ede9600e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:10.253693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9be2d5-5229-4a42-9e28-bf29ede9600e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:10.372975 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1163/2304] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:10 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:10.374335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:10.385122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33507e0-56f8-4f11-b214-2d0ceb81b447 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:10.432141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce32fc9-75be-4e63-9ed3-5cd7818260ae None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.433342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce32fc9-75be-4e63-9ed3-5cd7818260ae None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.433342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce32fc9-75be-4e63-9ed3-5cd7818260ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.448494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20528bcc-8331-4055-b2cf-bf0de5e9c927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:10.449172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528bcc-8331-4055-b2cf-bf0de5e9c927 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.449457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528bcc-8331-4055-b2cf-bf0de5e9c927 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.449735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528bcc-8331-4055-b2cf-bf0de5e9c927 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.450433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528bcc-8331-4055-b2cf-bf0de5e9c927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:10.451113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2ce32fc9-75be-4e63-9ed3-5cd7818260ae None None] MFA Rules not processed for user `8eedefd8d94047478663ceab8962d738`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:10.452365 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2ce32fc9-75be-4e63-9ed3-5cd7818260ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:10.452475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528bcc-8331-4055-b2cf-bf0de5e9c927 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:10.496035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:10.508105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1164/2305] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:10.517277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1142/2306] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:10 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:10.523345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce32fc9-75be-4e63-9ed3-5cd7818260ae None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:10.534412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce32fc9-75be-4e63-9ed3-5cd7818260ae None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:10.539705 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4ffbe-d288-4248-b75c-ab4ed7d74a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:10.540698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4ffbe-d288-4248-b75c-ab4ed7d74a69 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.541052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4ffbe-d288-4248-b75c-ab4ed7d74a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.541909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4ffbe-d288-4248-b75c-ab4ed7d74a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.542707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4ffbe-d288-4248-b75c-ab4ed7d74a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:10.544667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4ffbe-d288-4248-b75c-ab4ed7d74a69 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:10.558134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1165/2307] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:10.559309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20528bcc-8331-4055-b2cf-bf0de5e9c927 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:10.571260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20528bcc-8331-4055-b2cf-bf0de5e9c927 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:10.576380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db200f43-22fa-4557-9485-4bfef4294210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:10.577051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db200f43-22fa-4557-9485-4bfef4294210 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.577331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db200f43-22fa-4557-9485-4bfef4294210 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.577605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db200f43-22fa-4557-9485-4bfef4294210 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.578337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db200f43-22fa-4557-9485-4bfef4294210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:10.579979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db200f43-22fa-4557-9485-4bfef4294210 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:10.591590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1143/2308] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:10.595814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4ffbe-d288-4248-b75c-ab4ed7d74a69 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:10.606530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4ffbe-d288-4248-b75c-ab4ed7d74a69 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:10.611276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:10.612198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.612492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.612775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] PATH_INFO: `/v3/projects/8f06daf80ee3411ea26303b19e764337/users/ce4e96ab759148f3859c9f9d55ab8672/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.628259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f06daf80ee3411ea26303b19e764337, user_id=ce4e96ab759148f3859c9f9d55ab8672, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:10.631520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:10.740221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1166/2309] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:10 2022] PUT /identity/v3/projects/8f06daf80ee3411ea26303b19e764337/users/ce4e96ab759148f3859c9f9d55ab8672/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:10.751225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ad339-4c8a-47aa-8f5d-0cb7c6c0a7fe None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.751539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ad339-4c8a-47aa-8f5d-0cb7c6c0a7fe None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.751811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ad339-4c8a-47aa-8f5d-0cb7c6c0a7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.781989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c01ad339-4c8a-47aa-8f5d-0cb7c6c0a7fe None None] MFA Rules not processed for user `ce4e96ab759148f3859c9f9d55ab8672`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:10.783223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c01ad339-4c8a-47aa-8f5d-0cb7c6c0a7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:10.828169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:10.830169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:10.850188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1144/2310] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:10.861168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ad339-4c8a-47aa-8f5d-0cb7c6c0a7fe None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:10.862512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c01ad339-4c8a-47aa-8f5d-0cb7c6c0a7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:10.893782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1ac29-bbd6-4bc6-b6c3-1aee8ff302aa admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:10.897839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:10.898743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.899029 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.899312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.901236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:10.911215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:10.912696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:10.920890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:10.921220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1167/2311] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:05:10.937813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ad339-4c8a-47aa-8f5d-0cb7c6c0a7fe None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:10.944159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ed69e-3c23-4587-89cb-f0c05fed3d87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:10.944990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ed69e-3c23-4587-89cb-f0c05fed3d87 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:10.945275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ed69e-3c23-4587-89cb-f0c05fed3d87 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:10.945553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ed69e-3c23-4587-89cb-f0c05fed3d87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:10.954265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ed69e-3c23-4587-89cb-f0c05fed3d87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:10.955847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ed69e-3c23-4587-89cb-f0c05fed3d87 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:10.965117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1145/2312] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:05:11.468904 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:11.470462 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:11.548339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc59f16-b867-457d-9ea8-8065030b7815 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:11.556778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:11.558222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:11.558521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:11.559304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:11.562539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:11.576752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:11.583097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:11.590467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1168/2313] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:14.312968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ed69e-3c23-4587-89cb-f0c05fed3d87 service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:14.386501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ed69e-3c23-4587-89cb-f0c05fed3d87 service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:14.391515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:14.392451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:14.393456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:14.393456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] PATH_INFO: `/v3/users/ce4e96ab759148f3859c9f9d55ab8672` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:14.394223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce4e96ab759148f3859c9f9d55ab8672)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:14.396305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:14.493841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:14.504143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f77f69-ff03-44df-90b6-18f791328c50 service placement] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:14.509258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40199794-58a8-4c55-b37a-f93c63520bb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:14.509927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199794-58a8-4c55-b37a-f93c63520bb4 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:14.510241 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199794-58a8-4c55-b37a-f93c63520bb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:14.510522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40199794-58a8-4c55-b37a-f93c63520bb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:14.512516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-40199794-58a8-4c55-b37a-f93c63520bb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:14.523954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199794-58a8-4c55-b37a-f93c63520bb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:14.525546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40199794-58a8-4c55-b37a-f93c63520bb4 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:14.535331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1169/2314] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:05:14.600019 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce4e96ab759148f3859c9f9d55ab8672'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:14.668345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce4e96ab759148f3859c9f9d55ab8672'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:14.668975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce4e96ab759148f3859c9f9d55ab8672'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:14.676815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce4e96ab759148f3859c9f9d55ab8672'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:14.677482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce4e96ab759148f3859c9f9d55ab8672'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:14.697549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1146/2315] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:14 2022] DELETE /identity/v3/users/ce4e96ab759148f3859c9f9d55ab8672 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:14.699264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb97df0-518a-403b-97e8-98f96e3a6ab2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:14.699552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb97df0-518a-403b-97e8-98f96e3a6ab2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:14.699833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb97df0-518a-403b-97e8-98f96e3a6ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:14.719162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-dcb97df0-518a-403b-97e8-98f96e3a6ab2 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:14.720343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-dcb97df0-518a-403b-97e8-98f96e3a6ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:14.759277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:14.789390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1170/2316] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:14.799315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:14.802046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:14.864804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c8085a-a286-4dfe-8b48-e9682de8e117 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:14.869043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda48b5-a669-495d-83c5-a0651d99c574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:14.869715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda48b5-a669-495d-83c5-a0651d99c574 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:14.870003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda48b5-a669-495d-83c5-a0651d99c574 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:14.870282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda48b5-a669-495d-83c5-a0651d99c574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:14.881353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda48b5-a669-495d-83c5-a0651d99c574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:14.882821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda48b5-a669-495d-83c5-a0651d99c574 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:14.893817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1147/2317] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:15.338593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb97df0-518a-403b-97e8-98f96e3a6ab2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:15.340400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-dcb97df0-518a-403b-97e8-98f96e3a6ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:15.353431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb97df0-518a-403b-97e8-98f96e3a6ab2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:15.359624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:15.363508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:15.363508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:15.363508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] PATH_INFO: `/v3/projects/8f06daf80ee3411ea26303b19e764337` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:15.363508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f06daf80ee3411ea26303b19e764337)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:15.365938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:15.514430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f06daf80ee3411ea26303b19e764337', 'request_id': 'req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:15.594172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f06daf80ee3411ea26303b19e764337', 'request_id': 'req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:15.601749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f06daf80ee3411ea26303b19e764337', 'request_id': 'req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:15.613216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] The token cache is being invalidate because project 8f06daf80ee3411ea26303b19e764337 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:15.613506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:15.614780 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9589cd1e-b6ee-491e-a846-8ce416ffb7f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:15.616387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1171/2318] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:15 2022] DELETE /identity/v3/projects/8f06daf80ee3411ea26303b19e764337 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:15.621659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66069f-f78c-4bf0-b1cf-c4c0865c8923 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:15.622135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66069f-f78c-4bf0-b1cf-c4c0865c8923 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:15.622339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66069f-f78c-4bf0-b1cf-c4c0865c8923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:15.641653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2c66069f-f78c-4bf0-b1cf-c4c0865c8923 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:15.642790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2c66069f-f78c-4bf0-b1cf-c4c0865c8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:15.647210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:15.715364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1148/2319] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:15.724588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454bcee-18f1-40a5-ae7f-c1f896c191d7 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:15.725492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454bcee-18f1-40a5-ae7f-c1f896c191d7 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:15.725715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454bcee-18f1-40a5-ae7f-c1f896c191d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:15.746699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1454bcee-18f1-40a5-ae7f-c1f896c191d7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:15.747971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1454bcee-18f1-40a5-ae7f-c1f896c191d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:15.765230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1172/2320] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:15.769520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66069f-f78c-4bf0-b1cf-c4c0865c8923 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:15.779931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66069f-f78c-4bf0-b1cf-c4c0865c8923 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:15.787260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:15.787260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:15.787260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:15.787260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:15.787916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:15.787916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:15.789351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:15.804157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1149/2321] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:05:15.806825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454bcee-18f1-40a5-ae7f-c1f896c191d7 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:15.816569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454bcee-18f1-40a5-ae7f-c1f896c191d7 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:15.821932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fae0b-26af-4115-a269-c1e0a7e11cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:15.822677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fae0b-26af-4115-a269-c1e0a7e11cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:15.822963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fae0b-26af-4115-a269-c1e0a7e11cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:15.823241 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67fae0b-26af-4115-a269-c1e0a7e11cb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:15.823938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fae0b-26af-4115-a269-c1e0a7e11cb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:15.825933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67fae0b-26af-4115-a269-c1e0a7e11cb3 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:15.921829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1173/2322] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:15 2022] POST /identity/v3/projects => generated 407 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:15.926143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:15.937309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11209c8d-d24c-45f9-a764-055e9d614c46 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:15.996982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa6a66-c6ca-453d-bbbc-04f413bc677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:15.997686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6a66-c6ca-453d-bbbc-04f413bc677d admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:15.998088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6a66-c6ca-453d-bbbc-04f413bc677d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:15.998316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa6a66-c6ca-453d-bbbc-04f413bc677d admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:15.998950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa6a66-c6ca-453d-bbbc-04f413bc677d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.000879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa6a66-c6ca-453d-bbbc-04f413bc677d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.087324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1150/2323] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:15 2022] POST /identity/v3/users => generated 304 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:16.091623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fae0b-26af-4115-a269-c1e0a7e11cb3 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:16.103423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67fae0b-26af-4115-a269-c1e0a7e11cb3 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:16.108552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6394119-91c3-42cd-867b-7af7b1544be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:16.109421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6394119-91c3-42cd-867b-7af7b1544be9 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.109876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6394119-91c3-42cd-867b-7af7b1544be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.110157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6394119-91c3-42cd-867b-7af7b1544be9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.110897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6394119-91c3-42cd-867b-7af7b1544be9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.112588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6394119-91c3-42cd-867b-7af7b1544be9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.124584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1174/2324] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:16.128658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa6a66-c6ca-453d-bbbc-04f413bc677d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:16.139183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa6a66-c6ca-453d-bbbc-04f413bc677d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:16.143971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007304c-aa38-40f8-87e4-8cada2bae9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:16.147930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007304c-aa38-40f8-87e4-8cada2bae9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.147930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007304c-aa38-40f8-87e4-8cada2bae9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.147930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007304c-aa38-40f8-87e4-8cada2bae9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.147930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007304c-aa38-40f8-87e4-8cada2bae9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.148648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007304c-aa38-40f8-87e4-8cada2bae9a6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.162741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1151/2325] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:16.163914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6394119-91c3-42cd-867b-7af7b1544be9 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:16.174011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6394119-91c3-42cd-867b-7af7b1544be9 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:16.179632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104eeb-3c9d-40ab-987f-282f19b8a8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:16.180689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104eeb-3c9d-40ab-987f-282f19b8a8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.181034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104eeb-3c9d-40ab-987f-282f19b8a8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.181328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6104eeb-3c9d-40ab-987f-282f19b8a8ba admin admin] PATH_INFO: `/v3/projects/5521bf5e1f7d409785d9305ac74946d9/users/b2612ac914a54ad0a256ed5987f4ff5e/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.196905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6104eeb-3c9d-40ab-987f-282f19b8a8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5521bf5e1f7d409785d9305ac74946d9, user_id=b2612ac914a54ad0a256ed5987f4ff5e, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.199556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6104eeb-3c9d-40ab-987f-282f19b8a8ba admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.297289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1175/2326] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:16 2022] PUT /identity/v3/projects/5521bf5e1f7d409785d9305ac74946d9/users/b2612ac914a54ad0a256ed5987f4ff5e/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:16.309615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12661a16-af10-41a5-8c80-7ac3999e6325 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.309906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12661a16-af10-41a5-8c80-7ac3999e6325 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.310185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12661a16-af10-41a5-8c80-7ac3999e6325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.337449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-12661a16-af10-41a5-8c80-7ac3999e6325 None None] MFA Rules not processed for user `b2612ac914a54ad0a256ed5987f4ff5e`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:16.338666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-12661a16-af10-41a5-8c80-7ac3999e6325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:16.406769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1152/2327] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:16 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:16.411796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104eeb-3c9d-40ab-987f-282f19b8a8ba admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:16.481945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:16.486307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6104eeb-3c9d-40ab-987f-282f19b8a8ba admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:16.491744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577939ec-5873-4f64-abba-d10076602b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:16.492445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577939ec-5873-4f64-abba-d10076602b1b admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.492726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577939ec-5873-4f64-abba-d10076602b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.493001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577939ec-5873-4f64-abba-d10076602b1b admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.493689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577939ec-5873-4f64-abba-d10076602b1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.495623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577939ec-5873-4f64-abba-d10076602b1b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.590677 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1176/2328] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:16 2022] POST /identity/v3/projects => generated 409 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:16.594722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12661a16-af10-41a5-8c80-7ac3999e6325 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:16.602279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:16.606563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12661a16-af10-41a5-8c80-7ac3999e6325 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:16.661661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbecd8-7d25-4a02-890d-a39cb0c46161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:16.662581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbecd8-7d25-4a02-890d-a39cb0c46161 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.662684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbecd8-7d25-4a02-890d-a39cb0c46161 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.662915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbecd8-7d25-4a02-890d-a39cb0c46161 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.663628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbecd8-7d25-4a02-890d-a39cb0c46161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.665556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbecd8-7d25-4a02-890d-a39cb0c46161 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.757297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1153/2329] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:16 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:16.761398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577939ec-5873-4f64-abba-d10076602b1b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:16.771948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577939ec-5873-4f64-abba-d10076602b1b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:16.776926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed39cd1-8eb7-4671-bf69-f583f3592242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:16.777592 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed39cd1-8eb7-4671-bf69-f583f3592242 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.777877 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed39cd1-8eb7-4671-bf69-f583f3592242 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.778154 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed39cd1-8eb7-4671-bf69-f583f3592242 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.778881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed39cd1-8eb7-4671-bf69-f583f3592242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.780535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed39cd1-8eb7-4671-bf69-f583f3592242 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.792616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1177/2330] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:16.796873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbecd8-7d25-4a02-890d-a39cb0c46161 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:16.807590 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbecd8-7d25-4a02-890d-a39cb0c46161 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:16.812157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee429f-6ef3-4dd8-a33b-2279ded635c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:16.813032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee429f-6ef3-4dd8-a33b-2279ded635c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.813314 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee429f-6ef3-4dd8-a33b-2279ded635c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.813587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee429f-6ef3-4dd8-a33b-2279ded635c2 admin admin] PATH_INFO: `/v3/projects/fc033854ee5a498abd32cec19da00b95/users/0159c047ccd0495bbab49042785ef4db/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.828088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee429f-6ef3-4dd8-a33b-2279ded635c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc033854ee5a498abd32cec19da00b95, user_id=0159c047ccd0495bbab49042785ef4db, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.830778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee429f-6ef3-4dd8-a33b-2279ded635c2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.910291 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1154/2331] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:16 2022] PUT /identity/v3/projects/fc033854ee5a498abd32cec19da00b95/users/0159c047ccd0495bbab49042785ef4db/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:16.914789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed39cd1-8eb7-4671-bf69-f583f3592242 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:16.924790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed39cd1-8eb7-4671-bf69-f583f3592242 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:16.980124 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb20c6-e23c-4b75-a8fd-1a1873724df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:16.980819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb20c6-e23c-4b75-a8fd-1a1873724df1 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:16.981103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb20c6-e23c-4b75-a8fd-1a1873724df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:16.981382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb20c6-e23c-4b75-a8fd-1a1873724df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:16.982110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb20c6-e23c-4b75-a8fd-1a1873724df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:16.983718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb20c6-e23c-4b75-a8fd-1a1873724df1 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:16.993500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1178/2332] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:16.997908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee429f-6ef3-4dd8-a33b-2279ded635c2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:17.008303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee429f-6ef3-4dd8-a33b-2279ded635c2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:17.013061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:17.013934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:17.014215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:17.014487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] PATH_INFO: `/v3/projects/fc033854ee5a498abd32cec19da00b95/users/0159c047ccd0495bbab49042785ef4db/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:17.016460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc033854ee5a498abd32cec19da00b95, user_id=0159c047ccd0495bbab49042785ef4db, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:17.019130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:17.030290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1155/2333] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:16 2022] PUT /identity/v3/projects/fc033854ee5a498abd32cec19da00b95/users/0159c047ccd0495bbab49042785ef4db/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:17.041306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7381c-c62d-498a-87e9-6dc2f58ffe79 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:17.041595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7381c-c62d-498a-87e9-6dc2f58ffe79 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:17.041874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7381c-c62d-498a-87e9-6dc2f58ffe79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:17.068512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d1b7381c-c62d-498a-87e9-6dc2f58ffe79 None None] MFA Rules not processed for user `0159c047ccd0495bbab49042785ef4db`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:17.069715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d1b7381c-c62d-498a-87e9-6dc2f58ffe79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:17.137121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1179/2334] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:17 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:17.148619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:17.150399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:17.206818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b192d5e2-ff61-49bf-a92d-a8a61541ace7 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:17.211191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:17.211823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:17.212103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:17.212408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:17.221200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:17.222656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:17.232641 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1156/2335] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:17 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:17.609687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7381c-c62d-498a-87e9-6dc2f58ffe79 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:17.621773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7381c-c62d-498a-87e9-6dc2f58ffe79 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:17.624204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:17.624859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:17.625276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:17.625562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:17.678133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:17.686752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:17.688224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:17.697937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1180/2336] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:17 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:20.087551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:20.089039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:20.151211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:20.167125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8b6fd-fc8d-4d59-adbc-acb824e10fc6 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:20.172576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:20.173222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:20.173507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:20.173785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:20.175584 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:20.231580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:20.233124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:20.243442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1157/2337] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:24.950158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:24.952398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:24.970572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a752080-8f3f-40e6-96da-1d6a205282da service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:24.976600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc1874-a969-4c76-829d-b0fbfb67a9f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:24.977354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1874-a969-4c76-829d-b0fbfb67a9f3 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:24.977736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1874-a969-4c76-829d-b0fbfb67a9f3 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:24.978109 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc1874-a969-4c76-829d-b0fbfb67a9f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:24.979932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2ddc1874-a969-4c76-829d-b0fbfb67a9f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:24.993980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc1874-a969-4c76-829d-b0fbfb67a9f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:24.995559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc1874-a969-4c76-829d-b0fbfb67a9f3 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:25.007586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1181/2338] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:27.575227 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:27.637531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7caeb6-ca8b-4cfd-8c3a-2665d298eb6d service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:27.642133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:27.643014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:27.643300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:27.643576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] PATH_INFO: `/v3/users/b2612ac914a54ad0a256ed5987f4ff5e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:27.645066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2612ac914a54ad0a256ed5987f4ff5e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:27.647012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:27.810811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2612ac914a54ad0a256ed5987f4ff5e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:27.896539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2612ac914a54ad0a256ed5987f4ff5e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:27.897117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2612ac914a54ad0a256ed5987f4ff5e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:27.904442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2612ac914a54ad0a256ed5987f4ff5e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:27.905065 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2612ac914a54ad0a256ed5987f4ff5e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:27.922299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1158/2339] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:27 2022] DELETE /identity/v3/users/b2612ac914a54ad0a256ed5987f4ff5e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:27.927405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3a980-dea6-4ca5-8aaa-85b982e4bbc0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:27.927931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3a980-dea6-4ca5-8aaa-85b982e4bbc0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:27.928333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3a980-dea6-4ca5-8aaa-85b982e4bbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:27.945630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b2d3a980-dea6-4ca5-8aaa-85b982e4bbc0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:27.946910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b2d3a980-dea6-4ca5-8aaa-85b982e4bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:28.005423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1182/2340] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:28.015037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:28.016476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:28.035339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:28.073087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0fc52d-1b09-47ab-b06f-e8c469051d81 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:28.077816 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad702f8d-85fa-44a1-9571-47ce233bcd00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:28.078465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad702f8d-85fa-44a1-9571-47ce233bcd00 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:28.078749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad702f8d-85fa-44a1-9571-47ce233bcd00 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:28.079059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad702f8d-85fa-44a1-9571-47ce233bcd00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:28.088957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad702f8d-85fa-44a1-9571-47ce233bcd00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:28.090400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad702f8d-85fa-44a1-9571-47ce233bcd00 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:28.099746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1159/2341] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:28.530742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3a980-dea6-4ca5-8aaa-85b982e4bbc0 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:28.532316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b2d3a980-dea6-4ca5-8aaa-85b982e4bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:28.549911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3a980-dea6-4ca5-8aaa-85b982e4bbc0 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:28.556430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:28.557268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:28.557550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:28.557823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] PATH_INFO: `/v3/projects/5521bf5e1f7d409785d9305ac74946d9` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:28.559030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5521bf5e1f7d409785d9305ac74946d9)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:28.561045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:28.664060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5521bf5e1f7d409785d9305ac74946d9', 'request_id': 'req-877527cb-5035-4153-b355-26c03891ded4'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:28.743015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5521bf5e1f7d409785d9305ac74946d9', 'request_id': 'req-877527cb-5035-4153-b355-26c03891ded4'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:28.749659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5521bf5e1f7d409785d9305ac74946d9', 'request_id': 'req-877527cb-5035-4153-b355-26c03891ded4'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:28.771264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] The token cache is being invalidate because project 5521bf5e1f7d409785d9305ac74946d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:28.771264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:28.772496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:28.777344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1183/2342] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:28 2022] DELETE /identity/v3/projects/5521bf5e1f7d409785d9305ac74946d9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:28.778525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad702f8d-85fa-44a1-9571-47ce233bcd00 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:28.791526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad702f8d-85fa-44a1-9571-47ce233bcd00 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:28.864309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:28.865243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:28.865616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:28.865897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] PATH_INFO: `/v3/users/0159c047ccd0495bbab49042785ef4db` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:28.867174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0159c047ccd0495bbab49042785ef4db)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:28.869211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:29.088801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0159c047ccd0495bbab49042785ef4db'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:29.180202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0159c047ccd0495bbab49042785ef4db'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:29.181287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0159c047ccd0495bbab49042785ef4db'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:29.188404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0159c047ccd0495bbab49042785ef4db'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:29.189074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0159c047ccd0495bbab49042785ef4db'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:29.207074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1160/2343] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:28 2022] DELETE /identity/v3/users/0159c047ccd0495bbab49042785ef4db => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:29.716208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:29.717689 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:29.731800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877527cb-5035-4153-b355-26c03891ded4 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:29.811081 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:29.812639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:29.814579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:29.815472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:29.816333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:29.816652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] PATH_INFO: `/v3/projects/fc033854ee5a498abd32cec19da00b95` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:29.817892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc033854ee5a498abd32cec19da00b95)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:29.820472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:29.823435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:29.859600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:29.899846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba986a5e-d632-4661-bde6-00e4cd15c3c6 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:29.906154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817bfc9-cf7e-4f1b-8bd7-c8b1f4565b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:29.906971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bfc9-cf7e-4f1b-8bd7-c8b1f4565b82 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:29.907221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bfc9-cf7e-4f1b-8bd7-c8b1f4565b82 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:29.907498 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817bfc9-cf7e-4f1b-8bd7-c8b1f4565b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:29.910165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c817bfc9-cf7e-4f1b-8bd7-c8b1f4565b82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:29.924741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817bfc9-cf7e-4f1b-8bd7-c8b1f4565b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:29.926567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817bfc9-cf7e-4f1b-8bd7-c8b1f4565b82 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:29.940062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1161/2344] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:29.963971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc033854ee5a498abd32cec19da00b95', 'request_id': 'req-f2c22c7a-a6f8-4d70-b712-4220507ad277'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:30.040225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc033854ee5a498abd32cec19da00b95', 'request_id': 'req-f2c22c7a-a6f8-4d70-b712-4220507ad277'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:30.046936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc033854ee5a498abd32cec19da00b95', 'request_id': 'req-f2c22c7a-a6f8-4d70-b712-4220507ad277'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:30.060172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] The token cache is being invalidate because project fc033854ee5a498abd32cec19da00b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:30.060535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:30.061287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-f2c22c7a-a6f8-4d70-b712-4220507ad277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:30.063235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1184/2345] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:29 2022] DELETE /identity/v3/projects/fc033854ee5a498abd32cec19da00b95 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:30.068463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5ca1-5bb7-4d4b-bec2-f71275891cc1 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.068778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5ca1-5bb7-4d4b-bec2-f71275891cc1 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.069066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f5ca1-5bb7-4d4b-bec2-f71275891cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.099451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e75f5ca1-5bb7-4d4b-bec2-f71275891cc1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:30.101881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e75f5ca1-5bb7-4d4b-bec2-f71275891cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:30.173331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1162/2346] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:30.181537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc07286-3710-4909-8f4f-2eebe14da146 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.181810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc07286-3710-4909-8f4f-2eebe14da146 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.182140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc07286-3710-4909-8f4f-2eebe14da146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.201170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2cc07286-3710-4909-8f4f-2eebe14da146 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:30.202400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2cc07286-3710-4909-8f4f-2eebe14da146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:30.211397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1185/2347] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:30.215648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5ca1-5bb7-4d4b-bec2-f71275891cc1 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:30.227451 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f5ca1-5bb7-4d4b-bec2-f71275891cc1 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:30.242389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:30.243630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.243630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.243914 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.244873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:30.245247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:30.247053 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:30.260096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1163/2348] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:05:30.264656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc07286-3710-4909-8f4f-2eebe14da146 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:30.275136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc07286-3710-4909-8f4f-2eebe14da146 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:30.281010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408d6ad-fff3-4dfb-9ea5-d14832ed983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:30.282721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408d6ad-fff3-4dfb-9ea5-d14832ed983d admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.282721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408d6ad-fff3-4dfb-9ea5-d14832ed983d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.282933 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408d6ad-fff3-4dfb-9ea5-d14832ed983d admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.283710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408d6ad-fff3-4dfb-9ea5-d14832ed983d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:30.286255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408d6ad-fff3-4dfb-9ea5-d14832ed983d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:30.411836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1186/2349] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:30 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:30.416519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:30.427162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56505fbb-ddf6-4f86-8b94-5ac567fe6847 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:30.496858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893717c1-5f5e-44b7-9fcc-b398ec273c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:30.497640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893717c1-5f5e-44b7-9fcc-b398ec273c60 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.498026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893717c1-5f5e-44b7-9fcc-b398ec273c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.498789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893717c1-5f5e-44b7-9fcc-b398ec273c60 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.499163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893717c1-5f5e-44b7-9fcc-b398ec273c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:30.501506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893717c1-5f5e-44b7-9fcc-b398ec273c60 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:30.607411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1164/2350] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:30 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:30.611666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408d6ad-fff3-4dfb-9ea5-d14832ed983d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:30.623420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408d6ad-fff3-4dfb-9ea5-d14832ed983d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:30.628525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f74fd-4cea-43e5-bdf5-42b360934dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:30.629202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f74fd-4cea-43e5-bdf5-42b360934dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.629493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f74fd-4cea-43e5-bdf5-42b360934dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.629771 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f74fd-4cea-43e5-bdf5-42b360934dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.630526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f74fd-4cea-43e5-bdf5-42b360934dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:30.632883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f74fd-4cea-43e5-bdf5-42b360934dd5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:30.644067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1187/2351] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:30.648519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893717c1-5f5e-44b7-9fcc-b398ec273c60 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:30.658659 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893717c1-5f5e-44b7-9fcc-b398ec273c60 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:30.663980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb84543-c00a-4e33-9c19-4832fb3879e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:30.664615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb84543-c00a-4e33-9c19-4832fb3879e4 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.664903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb84543-c00a-4e33-9c19-4832fb3879e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.665187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb84543-c00a-4e33-9c19-4832fb3879e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.665919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb84543-c00a-4e33-9c19-4832fb3879e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:30.667525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb84543-c00a-4e33-9c19-4832fb3879e4 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:30.677895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1165/2352] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:30.682879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f74fd-4cea-43e5-bdf5-42b360934dd5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:30.695081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f74fd-4cea-43e5-bdf5-42b360934dd5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:30.699852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcce01-42cf-4d9b-b5d2-5d3841517703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:30.700779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcce01-42cf-4d9b-b5d2-5d3841517703 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.701059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcce01-42cf-4d9b-b5d2-5d3841517703 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.701333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcce01-42cf-4d9b-b5d2-5d3841517703 admin admin] PATH_INFO: `/v3/projects/81da3a21d8fa43fbb5c0487c0f681787/users/4b7a08a6dc1f48a5ac60f63a00230f88/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.717813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcce01-42cf-4d9b-b5d2-5d3841517703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81da3a21d8fa43fbb5c0487c0f681787, user_id=4b7a08a6dc1f48a5ac60f63a00230f88, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:30.720539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fcce01-42cf-4d9b-b5d2-5d3841517703 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:30.798406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1188/2353] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:30 2022] PUT /identity/v3/projects/81da3a21d8fa43fbb5c0487c0f681787/users/4b7a08a6dc1f48a5ac60f63a00230f88/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:30.810496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28791c-cc5c-4e5f-af67-8c711c19f7da None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.812567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28791c-cc5c-4e5f-af67-8c711c19f7da None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.812567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28791c-cc5c-4e5f-af67-8c711c19f7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.838629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9d28791c-cc5c-4e5f-af67-8c711c19f7da None None] MFA Rules not processed for user `4b7a08a6dc1f48a5ac60f63a00230f88`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:30.839851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9d28791c-cc5c-4e5f-af67-8c711c19f7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:30.857184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:30.905014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1166/2354] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:30.909565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcce01-42cf-4d9b-b5d2-5d3841517703 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:30.971649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcce01-42cf-4d9b-b5d2-5d3841517703 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:30.976416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5e492-840c-4a72-95eb-730412656542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:30.977073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5e492-840c-4a72-95eb-730412656542 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:30.977353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5e492-840c-4a72-95eb-730412656542 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:30.977627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5e492-840c-4a72-95eb-730412656542 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:30.978312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5e492-840c-4a72-95eb-730412656542 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:30.980259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5e492-840c-4a72-95eb-730412656542 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:31.077105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1189/2355] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:30 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:31.078486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28791c-cc5c-4e5f-af67-8c711c19f7da None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:31.089760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28791c-cc5c-4e5f-af67-8c711c19f7da None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:31.157807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19104087-2dde-4219-aea3-d79a88e49ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:31.158702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19104087-2dde-4219-aea3-d79a88e49ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:31.159055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19104087-2dde-4219-aea3-d79a88e49ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:31.159396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19104087-2dde-4219-aea3-d79a88e49ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:31.160185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19104087-2dde-4219-aea3-d79a88e49ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:31.162271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19104087-2dde-4219-aea3-d79a88e49ca7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:31.268372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1167/2356] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:31 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:31.271384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5e492-840c-4a72-95eb-730412656542 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:31.284857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5e492-840c-4a72-95eb-730412656542 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:31.289976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d9cb9-480a-48bb-a24c-a0b028494467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:31.291080 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d9cb9-480a-48bb-a24c-a0b028494467 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:31.291371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d9cb9-480a-48bb-a24c-a0b028494467 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:31.291652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d9cb9-480a-48bb-a24c-a0b028494467 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:31.292437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d9cb9-480a-48bb-a24c-a0b028494467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:31.294912 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2d9cb9-480a-48bb-a24c-a0b028494467 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:31.309006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1190/2357] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:31.313239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19104087-2dde-4219-aea3-d79a88e49ca7 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:31.324940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19104087-2dde-4219-aea3-d79a88e49ca7 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:31.329613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338505a-40f7-464a-86e1-ef094914eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:31.330508 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338505a-40f7-464a-86e1-ef094914eaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:31.330924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338505a-40f7-464a-86e1-ef094914eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:31.331211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5338505a-40f7-464a-86e1-ef094914eaa7 admin admin] PATH_INFO: `/v3/projects/244d7f7ba8c04b539e12d91517910379/users/15d8c9469cc14c37bc3159d709c07f24/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:31.347269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338505a-40f7-464a-86e1-ef094914eaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244d7f7ba8c04b539e12d91517910379, user_id=15d8c9469cc14c37bc3159d709c07f24, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:31.350001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5338505a-40f7-464a-86e1-ef094914eaa7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:31.423244 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1168/2358] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:31 2022] PUT /identity/v3/projects/244d7f7ba8c04b539e12d91517910379/users/15d8c9469cc14c37bc3159d709c07f24/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:31.427209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d9cb9-480a-48bb-a24c-a0b028494467 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:31.433170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:31.438726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d9cb9-480a-48bb-a24c-a0b028494467 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:31.498368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b654b8-5676-400f-8bab-9213af67dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:31.499045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b654b8-5676-400f-8bab-9213af67dffb admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:31.499327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b654b8-5676-400f-8bab-9213af67dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:31.499603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b654b8-5676-400f-8bab-9213af67dffb admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:31.500400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b654b8-5676-400f-8bab-9213af67dffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:31.502290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b654b8-5676-400f-8bab-9213af67dffb admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:31.516908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1191/2359] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:31.518184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338505a-40f7-464a-86e1-ef094914eaa7 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:31.529739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5338505a-40f7-464a-86e1-ef094914eaa7 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:31.534602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:31.535499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:31.535785 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:31.536066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] PATH_INFO: `/v3/projects/244d7f7ba8c04b539e12d91517910379/users/15d8c9469cc14c37bc3159d709c07f24/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:31.538844 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] RBAC: Authorizing `identity:create_grant(project_id=244d7f7ba8c04b539e12d91517910379, user_id=15d8c9469cc14c37bc3159d709c07f24, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:31.541677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:31.552959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1169/2360] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:31 2022] PUT /identity/v3/projects/244d7f7ba8c04b539e12d91517910379/users/15d8c9469cc14c37bc3159d709c07f24/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:31.563851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8877ed-9f32-4453-8826-99aa75869b0e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:31.567464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8877ed-9f32-4453-8826-99aa75869b0e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:31.567464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8877ed-9f32-4453-8826-99aa75869b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:31.592848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-5e8877ed-9f32-4453-8826-99aa75869b0e None None] MFA Rules not processed for user `15d8c9469cc14c37bc3159d709c07f24`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:31.594066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5e8877ed-9f32-4453-8826-99aa75869b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:31.671391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1192/2361] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:31.681280 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:31.683033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:31.744924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b889622f-9f90-4185-a3b1-a48392921476 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:31.749083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24859ad-d5da-4146-a733-d78601ca79bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:31.749744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24859ad-d5da-4146-a733-d78601ca79bf service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:31.750030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24859ad-d5da-4146-a733-d78601ca79bf service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:31.750306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24859ad-d5da-4146-a733-d78601ca79bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:31.761601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24859ad-d5da-4146-a733-d78601ca79bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:31.763121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24859ad-d5da-4146-a733-d78601ca79bf service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:31.775148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1170/2362] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:32.303723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8877ed-9f32-4453-8826-99aa75869b0e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:32.372412 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8877ed-9f32-4453-8826-99aa75869b0e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:32.377080 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:32.377883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:32.378165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:32.378436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] PATH_INFO: `/v3/users/4b7a08a6dc1f48a5ac60f63a00230f88` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:32.379637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7a08a6dc1f48a5ac60f63a00230f88)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:32.382220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:32.503820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:32.611193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7a08a6dc1f48a5ac60f63a00230f88'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:32.716597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7a08a6dc1f48a5ac60f63a00230f88'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:32.717239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7a08a6dc1f48a5ac60f63a00230f88'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:32.724428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7a08a6dc1f48a5ac60f63a00230f88'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:32.725596 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7a08a6dc1f48a5ac60f63a00230f88'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:32.743330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1193/2363] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:32 2022] DELETE /identity/v3/users/4b7a08a6dc1f48a5ac60f63a00230f88 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:32.748492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2db0a-b608-491b-ac20-bc6b1fbb73a8 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:32.748962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2db0a-b608-491b-ac20-bc6b1fbb73a8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:32.749085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2db0a-b608-491b-ac20-bc6b1fbb73a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:32.768203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-37f2db0a-b608-491b-ac20-bc6b1fbb73a8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:32.769399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-37f2db0a-b608-491b-ac20-bc6b1fbb73a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:32.778215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:32.829173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1171/2364] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:32.838899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:32.840362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:32.897536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0414f-9948-453d-9684-93720bb45076 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:32.901334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31be150-ee3d-4188-8219-7c1894dea0cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:32.904058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31be150-ee3d-4188-8219-7c1894dea0cf service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:32.904058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31be150-ee3d-4188-8219-7c1894dea0cf service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:32.904058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31be150-ee3d-4188-8219-7c1894dea0cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:32.911603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31be150-ee3d-4188-8219-7c1894dea0cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:32.913602 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31be150-ee3d-4188-8219-7c1894dea0cf service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:32.923247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1194/2365] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:33.336070 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2db0a-b608-491b-ac20-bc6b1fbb73a8 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:33.337511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-37f2db0a-b608-491b-ac20-bc6b1fbb73a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:33.349871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2db0a-b608-491b-ac20-bc6b1fbb73a8 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:33.355071 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:33.355972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:33.356288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:33.356570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] PATH_INFO: `/v3/projects/81da3a21d8fa43fbb5c0487c0f681787` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:33.361386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81da3a21d8fa43fbb5c0487c0f681787)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:33.361386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:33.460214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81da3a21d8fa43fbb5c0487c0f681787', 'request_id': 'req-73f88946-e32e-492e-b007-9311a2fe2ae9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.540095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81da3a21d8fa43fbb5c0487c0f681787', 'request_id': 'req-73f88946-e32e-492e-b007-9311a2fe2ae9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.546562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81da3a21d8fa43fbb5c0487c0f681787', 'request_id': 'req-73f88946-e32e-492e-b007-9311a2fe2ae9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.558082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] The token cache is being invalidate because project 81da3a21d8fa43fbb5c0487c0f681787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:33.558368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.559043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.560630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1172/2366] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:33 2022] DELETE /identity/v3/projects/81da3a21d8fa43fbb5c0487c0f681787 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:33.564728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31be150-ee3d-4188-8219-7c1894dea0cf service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:33.575261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31be150-ee3d-4188-8219-7c1894dea0cf service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:33.634992 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:33.635801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:33.636088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:33.636405 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] PATH_INFO: `/v3/users/15d8c9469cc14c37bc3159d709c07f24` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:33.637671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d8c9469cc14c37bc3159d709c07f24)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:33.639584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:33.813529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d8c9469cc14c37bc3159d709c07f24'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.815303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:33.925210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d8c9469cc14c37bc3159d709c07f24'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.925817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d8c9469cc14c37bc3159d709c07f24'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.933282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d8c9469cc14c37bc3159d709c07f24'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.933990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67070cf2-a146-44f0-aeb5-167aca448fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d8c9469cc14c37bc3159d709c07f24'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:33.952376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1195/2367] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:33 2022] DELETE /identity/v3/users/15d8c9469cc14c37bc3159d709c07f24 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:34.434246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:34.435863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:34.451433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f88946-e32e-492e-b007-9311a2fe2ae9 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:34.510686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:34.511614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:34.511899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:34.512201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] PATH_INFO: `/v3/projects/244d7f7ba8c04b539e12d91517910379` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:34.513589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244d7f7ba8c04b539e12d91517910379)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:34.515598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:34.627912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:34.636801 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244d7f7ba8c04b539e12d91517910379', 'request_id': 'req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:34.757077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244d7f7ba8c04b539e12d91517910379', 'request_id': 'req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:34.775676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244d7f7ba8c04b539e12d91517910379', 'request_id': 'req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:34.789252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] The token cache is being invalidate because project 244d7f7ba8c04b539e12d91517910379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:34.789252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:34.790001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ff578e3b-d5eb-46cf-aa60-e7c4f2e3a5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:34.791666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1173/2368] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:34 2022] DELETE /identity/v3/projects/244d7f7ba8c04b539e12d91517910379 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:34.796893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1cf2f-f774-4ff2-b3b0-065dc4ddecdf None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:34.797423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1cf2f-f774-4ff2-b3b0-065dc4ddecdf None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:34.797666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1cf2f-f774-4ff2-b3b0-065dc4ddecdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:34.815526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-93c1cf2f-f774-4ff2-b3b0-065dc4ddecdf None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:34.817028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-93c1cf2f-f774-4ff2-b3b0-065dc4ddecdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:34.888438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1196/2369] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:34.896059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1a92-7c4f-43de-8b5f-10e649448329 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:34.896389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1a92-7c4f-43de-8b5f-10e649448329 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:34.896666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a1a92-7c4f-43de-8b5f-10e649448329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:34.915853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-429a1a92-7c4f-43de-8b5f-10e649448329 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:34.917130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-429a1a92-7c4f-43de-8b5f-10e649448329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:34.924882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1174/2370] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:34.930892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1cf2f-f774-4ff2-b3b0-065dc4ddecdf None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:34.940787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1cf2f-f774-4ff2-b3b0-065dc4ddecdf None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:34.945613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:34.946311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:34.946594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:34.946892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:34.947681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:05:34.948033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:34.949666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:34.964528 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1197/2371] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:05:34.968913 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a1a92-7c4f-43de-8b5f-10e649448329 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:34.978962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a1a92-7c4f-43de-8b5f-10e649448329 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:34.983705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a433ec8-fc18-4c7e-a123-887ca2657b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:34.985271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a433ec8-fc18-4c7e-a123-887ca2657b74 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:34.985271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a433ec8-fc18-4c7e-a123-887ca2657b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:34.985271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a433ec8-fc18-4c7e-a123-887ca2657b74 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:34.985605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a433ec8-fc18-4c7e-a123-887ca2657b74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:34.987506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a433ec8-fc18-4c7e-a123-887ca2657b74 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:35.090303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1175/2372] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:34 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:35.094636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:35.105782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834785-2322-48fd-b34e-896403e2e68d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:35.161334 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54867429-b828-4f1a-b82d-7aa008376fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:35.162000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54867429-b828-4f1a-b82d-7aa008376fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:35.162279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54867429-b828-4f1a-b82d-7aa008376fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:35.162550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54867429-b828-4f1a-b82d-7aa008376fc6 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:35.163268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54867429-b828-4f1a-b82d-7aa008376fc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:35.165275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54867429-b828-4f1a-b82d-7aa008376fc6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:35.280790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1198/2373] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:35 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:35.285158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a433ec8-fc18-4c7e-a123-887ca2657b74 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:35.295936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a433ec8-fc18-4c7e-a123-887ca2657b74 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:35.300539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf2d93-04d3-446c-9f24-7318eeb47a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:35.301196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf2d93-04d3-446c-9f24-7318eeb47a71 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:35.301482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf2d93-04d3-446c-9f24-7318eeb47a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:35.301757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf2d93-04d3-446c-9f24-7318eeb47a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:35.302541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf2d93-04d3-446c-9f24-7318eeb47a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:35.304255 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf2d93-04d3-446c-9f24-7318eeb47a71 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:35.314699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1176/2374] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:35.318953 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54867429-b828-4f1a-b82d-7aa008376fc6 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:35.329827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54867429-b828-4f1a-b82d-7aa008376fc6 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:35.334538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaca3ce-7c57-4b66-bbe6-0751df8bfd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:35.335914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaca3ce-7c57-4b66-bbe6-0751df8bfd8a admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:35.336233 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaca3ce-7c57-4b66-bbe6-0751df8bfd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:35.336519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaca3ce-7c57-4b66-bbe6-0751df8bfd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:35.337273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaca3ce-7c57-4b66-bbe6-0751df8bfd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:35.338918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaca3ce-7c57-4b66-bbe6-0751df8bfd8a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:35.348943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1199/2375] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:35.353209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf2d93-04d3-446c-9f24-7318eeb47a71 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:35.363233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf2d93-04d3-446c-9f24-7318eeb47a71 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:35.367892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:35.368809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:35.369096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:35.369377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] PATH_INFO: `/v3/projects/eccbd6e74d2a451aaada3c14ca814085/users/541db58d13ca430491a5b3002d435515/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:35.384237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eccbd6e74d2a451aaada3c14ca814085, user_id=541db58d13ca430491a5b3002d435515, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:35.387028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:35.479403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1177/2376] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:35 2022] PUT /identity/v3/projects/eccbd6e74d2a451aaada3c14ca814085/users/541db58d13ca430491a5b3002d435515/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:35.484713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93c935-8b73-4807-a833-4803eb2598a0 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:35.485034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93c935-8b73-4807-a833-4803eb2598a0 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:35.485320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93c935-8b73-4807-a833-4803eb2598a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:35.503614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9f93c935-8b73-4807-a833-4803eb2598a0 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:35.504874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9f93c935-8b73-4807-a833-4803eb2598a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:35.530981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:35.563527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1200/2377] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:35.574017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:35.575752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:35.631213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db91039-7db4-49cf-bc16-8a2da39e89bd admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:35.635837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4d0b2-3aba-45e3-80c1-c1b9bcc14369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:35.636527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4d0b2-3aba-45e3-80c1-c1b9bcc14369 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:35.636818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4d0b2-3aba-45e3-80c1-c1b9bcc14369 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:35.637101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4d0b2-3aba-45e3-80c1-c1b9bcc14369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:35.647006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4d0b2-3aba-45e3-80c1-c1b9bcc14369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:35.648965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4d0b2-3aba-45e3-80c1-c1b9bcc14369 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:35.658841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1178/2378] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:36.962979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71596cc-9aff-45ee-98f9-2d46353e0424 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:36.963537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71596cc-9aff-45ee-98f9-2d46353e0424 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:36.963537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71596cc-9aff-45ee-98f9-2d46353e0424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:36.992833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b71596cc-9aff-45ee-98f9-2d46353e0424 None None] MFA Rules not processed for user `541db58d13ca430491a5b3002d435515`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:36.994067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b71596cc-9aff-45ee-98f9-2d46353e0424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:37.073533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1201/2379] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:37.077749 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4d0b2-3aba-45e3-80c1-c1b9bcc14369 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.089485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4d0b2-3aba-45e3-80c1-c1b9bcc14369 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:37.095255 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa44f-9344-48dc-bc30-31395fb1dd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:37.096136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa44f-9344-48dc-bc30-31395fb1dd03 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.096460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa44f-9344-48dc-bc30-31395fb1dd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.096739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfa44f-9344-48dc-bc30-31395fb1dd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.097674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfa44f-9344-48dc-bc30-31395fb1dd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:37.099594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfa44f-9344-48dc-bc30-31395fb1dd03 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:37.218580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1179/2380] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:05:37 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:05:37.222712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71596cc-9aff-45ee-98f9-2d46353e0424 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.233952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71596cc-9aff-45ee-98f9-2d46353e0424 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:37.292041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8144b-571a-4c67-a4ae-1e5ca5b3b2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:37.292813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8144b-571a-4c67-a4ae-1e5ca5b3b2b4 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.293100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8144b-571a-4c67-a4ae-1e5ca5b3b2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.293394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8144b-571a-4c67-a4ae-1e5ca5b3b2b4 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.294153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8144b-571a-4c67-a4ae-1e5ca5b3b2b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:37.296230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8144b-571a-4c67-a4ae-1e5ca5b3b2b4 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:37.409713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1202/2381] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:05:37 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:05:37.414558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa44f-9344-48dc-bc30-31395fb1dd03 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.424927 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfa44f-9344-48dc-bc30-31395fb1dd03 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:37.430120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e094c-c08f-45cc-a393-4ca5dcb83dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:37.430779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e094c-c08f-45cc-a393-4ca5dcb83dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.431062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e094c-c08f-45cc-a393-4ca5dcb83dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.431339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e094c-c08f-45cc-a393-4ca5dcb83dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.432066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e094c-c08f-45cc-a393-4ca5dcb83dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:37.433689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e094c-c08f-45cc-a393-4ca5dcb83dc4 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:37.443991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1180/2382] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:05:37.448196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8144b-571a-4c67-a4ae-1e5ca5b3b2b4 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.458111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8144b-571a-4c67-a4ae-1e5ca5b3b2b4 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:37.462893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bd211-806a-48e9-9940-659ac6ec6aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:37.463770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd211-806a-48e9-9940-659ac6ec6aad admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.464054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd211-806a-48e9-9940-659ac6ec6aad admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.467945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd211-806a-48e9-9940-659ac6ec6aad admin admin] PATH_INFO: `/v3/projects/b96bdc58031c4b99bdc90af653b93e04/users/bd7b2cec9af847e0868f67527bfeb615/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.480414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1bd211-806a-48e9-9940-659ac6ec6aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96bdc58031c4b99bdc90af653b93e04, user_id=bd7b2cec9af847e0868f67527bfeb615, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:37.483071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1bd211-806a-48e9-9940-659ac6ec6aad admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:37.559276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1203/2383] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:37 2022] PUT /identity/v3/projects/b96bdc58031c4b99bdc90af653b93e04/users/bd7b2cec9af847e0868f67527bfeb615/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:37.563365 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e094c-c08f-45cc-a393-4ca5dcb83dc4 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.573867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e094c-c08f-45cc-a393-4ca5dcb83dc4 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:37.606739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:37.637095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ec035-7692-4c34-af61-7da46c04c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:37.637798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ec035-7692-4c34-af61-7da46c04c967 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.638085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ec035-7692-4c34-af61-7da46c04c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.638364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ec035-7692-4c34-af61-7da46c04c967 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.639147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ec035-7692-4c34-af61-7da46c04c967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:37.640780 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ec035-7692-4c34-af61-7da46c04c967 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:37.651204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1181/2384] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:05:37.655326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bd211-806a-48e9-9940-659ac6ec6aad admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:37.666532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bd211-806a-48e9-9940-659ac6ec6aad admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:37.671337 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faef4a-4810-4015-95e5-c4087872a06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:37.672263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faef4a-4810-4015-95e5-c4087872a06e admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:37.672544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faef4a-4810-4015-95e5-c4087872a06e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:37.672817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faef4a-4810-4015-95e5-c4087872a06e admin admin] PATH_INFO: `/v3/projects/b96bdc58031c4b99bdc90af653b93e04/users/bd7b2cec9af847e0868f67527bfeb615/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:37.674825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faef4a-4810-4015-95e5-c4087872a06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96bdc58031c4b99bdc90af653b93e04, user_id=bd7b2cec9af847e0868f67527bfeb615, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:37.677564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faef4a-4810-4015-95e5-c4087872a06e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:37.689361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1204/2385] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:05:37 2022] PUT /identity/v3/projects/b96bdc58031c4b99bdc90af653b93e04/users/bd7b2cec9af847e0868f67527bfeb615/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:38.845779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce997-20bf-484e-bb56-688b75f65afe None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:38.846636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce997-20bf-484e-bb56-688b75f65afe None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:38.846943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce997-20bf-484e-bb56-688b75f65afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:38.866175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-fafce997-20bf-484e-bb56-688b75f65afe None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:38.868586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fafce997-20bf-484e-bb56-688b75f65afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:38.933385 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1182/2386] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:05:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:05:38.970136 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19968f-c697-4666-9435-14c182233dd3 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:38.970671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19968f-c697-4666-9435-14c182233dd3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:38.971206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19968f-c697-4666-9435-14c182233dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:38.997161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:39.000284 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fa19968f-c697-4666-9435-14c182233dd3 None None] MFA Rules not processed for user `bd7b2cec9af847e0868f67527bfeb615`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:05:39.001669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fa19968f-c697-4666-9435-14c182233dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:39.079776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1205/2387] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:05:39.091161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce997-20bf-484e-bb56-688b75f65afe None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:39.092739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fafce997-20bf-484e-bb56-688b75f65afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:39.115757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19968f-c697-4666-9435-14c182233dd3 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:39.118629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fa19968f-c697-4666-9435-14c182233dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:39.166755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce997-20bf-484e-bb56-688b75f65afe None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:39.171514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd63cd-ff6b-4515-bc4f-241448a6e3d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:39.172381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd63cd-ff6b-4515-bc4f-241448a6e3d6 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.172757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd63cd-ff6b-4515-bc4f-241448a6e3d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.173288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd63cd-ff6b-4515-bc4f-241448a6e3d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.202289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19968f-c697-4666-9435-14c182233dd3 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:39.207772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca45be-dc83-4295-b1b5-0695fc130889 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:39.208471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca45be-dc83-4295-b1b5-0695fc130889 service swift] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:39.208759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca45be-dc83-4295-b1b5-0695fc130889 service swift] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:39.209040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ca45be-dc83-4295-b1b5-0695fc130889 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:39.218986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca45be-dc83-4295-b1b5-0695fc130889 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:39.220632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ca45be-dc83-4295-b1b5-0695fc130889 service swift] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:39.229758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1206/2388] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:05:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 06:05:39.251165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd63cd-ff6b-4515-bc4f-241448a6e3d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:39.259257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd63cd-ff6b-4515-bc4f-241448a6e3d6 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:39.274041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1183/2389] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:05:41.118252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca45be-dc83-4295-b1b5-0695fc130889 service swift] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:41.127807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ca45be-dc83-4295-b1b5-0695fc130889 service swift] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:41.136300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:41.137039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:41.137331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:41.137609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:41.149482 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:41.154715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:41.163441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1207/2390] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:49.268906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=87 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:51.158721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=66 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:56.404598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd63cd-ff6b-4515-bc4f-241448a6e3d6 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:56.465185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd63cd-ff6b-4515-bc4f-241448a6e3d6 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:56.468824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:56.469480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:56.469762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:56.470032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:56.471815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:56.484269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:56.485805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:56.495230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1184/2391] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:57.107302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:57.108884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:57.197431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ea659-f7f7-4978-a239-544bd3107785 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:57.197942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:57.198601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:57.198886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:57.199162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:57.201021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:57.213647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:57.215225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:57.227065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1208/2392] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:05:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:05:58.025228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:58.099779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc123fa0-f282-479d-a290-ee44a5876fda service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:58.104853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:58.105735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:58.106018 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:58.106294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] PATH_INFO: `/v3/users/541db58d13ca430491a5b3002d435515` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:58.107485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=541db58d13ca430491a5b3002d435515)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:58.109525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:58.346971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541db58d13ca430491a5b3002d435515'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.420507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541db58d13ca430491a5b3002d435515'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.421325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541db58d13ca430491a5b3002d435515'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.430640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541db58d13ca430491a5b3002d435515'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.435390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541db58d13ca430491a5b3002d435515'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:58.458138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1185/2393] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:58 2022] DELETE /identity/v3/users/541db58d13ca430491a5b3002d435515 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:05:58.732263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:58.744079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a47138-2df8-401c-81c1-5d4166ebf9d2 service placement] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:58.814244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:58.815110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:58.815399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:58.815693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] PATH_INFO: `/v3/projects/eccbd6e74d2a451aaada3c14ca814085` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:58.816944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eccbd6e74d2a451aaada3c14ca814085)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:58.818880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:58.937837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eccbd6e74d2a451aaada3c14ca814085', 'request_id': 'req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.024773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eccbd6e74d2a451aaada3c14ca814085', 'request_id': 'req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.031246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eccbd6e74d2a451aaada3c14ca814085', 'request_id': 'req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.048380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] The token cache is being invalidate because project eccbd6e74d2a451aaada3c14ca814085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:59.048694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.049369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.050905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1209/2394] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:58 2022] DELETE /identity/v3/projects/eccbd6e74d2a451aaada3c14ca814085 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:59.055498 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.056996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:59.070482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a23fe-5cc7-4165-a08a-9e11e76bb9fb admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.090625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:59.132635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.135194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.135689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.136236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] PATH_INFO: `/v3/users/bd7b2cec9af847e0868f67527bfeb615` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.138984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7b2cec9af847e0868f67527bfeb615)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.142106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.345984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7b2cec9af847e0868f67527bfeb615'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.401116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7b2cec9af847e0868f67527bfeb615'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.401665 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7b2cec9af847e0868f67527bfeb615'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.415719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7b2cec9af847e0868f67527bfeb615'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.415719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-ae4c030e-5ef8-42ad-a927-045e5f6def42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7b2cec9af847e0868f67527bfeb615'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.427895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1186/2395] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:05:59 2022] DELETE /identity/v3/users/bd7b2cec9af847e0868f67527bfeb615 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:59.683123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:05:59.683979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:05:59.698901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b05ee-61ad-4350-bb80-a8dba1d9afa9 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:05:59.719005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:05:59.759654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:05:59.760554 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.760878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.761158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] PATH_INFO: `/v3/projects/b96bdc58031c4b99bdc90af653b93e04` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:05:59.762381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96bdc58031c4b99bdc90af653b93e04)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:05:59.764378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:05:59.881362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96bdc58031c4b99bdc90af653b93e04', 'request_id': 'req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.971611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96bdc58031c4b99bdc90af653b93e04', 'request_id': 'req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.978717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96bdc58031c4b99bdc90af653b93e04', 'request_id': 'req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.989867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] The token cache is being invalidate because project b96bdc58031c4b99bdc90af653b93e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:05:59.990164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.990788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6bbb0bf8-4733-4fe4-b7a2-6c7e53930896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:05:59.993104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1210/2396] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:05:59 2022] DELETE /identity/v3/projects/b96bdc58031c4b99bdc90af653b93e04 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:05:59.998013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0497fb7-6c61-4b7a-b673-d4723bd7a240 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:05:59.998300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0497fb7-6c61-4b7a-b673-d4723bd7a240 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:05:59.998578 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0497fb7-6c61-4b7a-b673-d4723bd7a240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.016240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d0497fb7-6c61-4b7a-b673-d4723bd7a240 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:00.017413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d0497fb7-6c61-4b7a-b673-d4723bd7a240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.075946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1187/2397] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:00.084950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bbbcb-b55a-47c1-b079-f05bfeaefbd1 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.085246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bbbcb-b55a-47c1-b079-f05bfeaefbd1 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.085527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bbbcb-b55a-47c1-b079-f05bfeaefbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.104497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-976bbbcb-b55a-47c1-b079-f05bfeaefbd1 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:00.105700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-976bbbcb-b55a-47c1-b079-f05bfeaefbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.113636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1211/2398] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:00.118448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0497fb7-6c61-4b7a-b673-d4723bd7a240 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.127928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0497fb7-6c61-4b7a-b673-d4723bd7a240 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.132452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.133135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.133414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.133689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.134410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:00.134715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.136315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.147570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1188/2399] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:00.152278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bbbcb-b55a-47c1-b079-f05bfeaefbd1 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.161498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bbbcb-b55a-47c1-b079-f05bfeaefbd1 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.167217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e24d6-36ab-4277-a3ca-0eaa9b1c3fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.167872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e24d6-36ab-4277-a3ca-0eaa9b1c3fce admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.168187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e24d6-36ab-4277-a3ca-0eaa9b1c3fce admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.168466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e24d6-36ab-4277-a3ca-0eaa9b1c3fce admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.169150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e24d6-36ab-4277-a3ca-0eaa9b1c3fce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.171059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5e24d6-36ab-4277-a3ca-0eaa9b1c3fce admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.246935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1212/2400] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:00.251011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.261191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab60375-a9ce-449d-bebe-3ec3d1fdcc78 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.286520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:00.318134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f33312-4682-4509-8db0-bb15f9c648fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.318814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33312-4682-4509-8db0-bb15f9c648fe admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.319097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33312-4682-4509-8db0-bb15f9c648fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.319371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f33312-4682-4509-8db0-bb15f9c648fe admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.320058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f33312-4682-4509-8db0-bb15f9c648fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.322049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f33312-4682-4509-8db0-bb15f9c648fe admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.390079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1189/2401] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:00.394471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e24d6-36ab-4277-a3ca-0eaa9b1c3fce admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.408069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e24d6-36ab-4277-a3ca-0eaa9b1c3fce admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.413119 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7dcb0-fe6a-4de6-a81e-fe0c47f5850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.413872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7dcb0-fe6a-4de6-a81e-fe0c47f5850f admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.414792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7dcb0-fe6a-4de6-a81e-fe0c47f5850f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.415320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7dcb0-fe6a-4de6-a81e-fe0c47f5850f admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.416403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7dcb0-fe6a-4de6-a81e-fe0c47f5850f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.418178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7dcb0-fe6a-4de6-a81e-fe0c47f5850f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.430035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1213/2402] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:00.440404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f33312-4682-4509-8db0-bb15f9c648fe admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.452847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f33312-4682-4509-8db0-bb15f9c648fe admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.458569 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b7b72-5407-4ab1-a474-3b532f156742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.459251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b7b72-5407-4ab1-a474-3b532f156742 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.459540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b7b72-5407-4ab1-a474-3b532f156742 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.459821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b7b72-5407-4ab1-a474-3b532f156742 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.460589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b7b72-5407-4ab1-a474-3b532f156742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.462925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b7b72-5407-4ab1-a474-3b532f156742 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.485951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1190/2403] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:00.487165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7dcb0-fe6a-4de6-a81e-fe0c47f5850f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.500626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7dcb0-fe6a-4de6-a81e-fe0c47f5850f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.502817 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.503779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.504149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.504505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] PATH_INFO: `/v3/projects/b75327cab649456ca56a425eacbc4377/users/d633030694cb49a4a478f1f7a6d21c2b/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.524411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75327cab649456ca56a425eacbc4377, user_id=d633030694cb49a4a478f1f7a6d21c2b, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.529037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.636512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1214/2404] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:00 2022] PUT /identity/v3/projects/b75327cab649456ca56a425eacbc4377/users/d633030694cb49a4a478f1f7a6d21c2b/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:00.647229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98a52c-80cc-4878-8ec8-ca404643f90e None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.647515 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98a52c-80cc-4878-8ec8-ca404643f90e None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.647813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98a52c-80cc-4878-8ec8-ca404643f90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.679181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ac98a52c-80cc-4878-8ec8-ca404643f90e None None] MFA Rules not processed for user `d633030694cb49a4a478f1f7a6d21c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:00.680858 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ac98a52c-80cc-4878-8ec8-ca404643f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.752301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1191/2405] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:00.763899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.766219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:00.819749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:00.825252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59d182-a6e2-4bd1-ad26-430556c01403 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.828963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc5cd2-8790-4bd9-9881-53b4d48afabd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.829610 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5cd2-8790-4bd9-9881-53b4d48afabd service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.829911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5cd2-8790-4bd9-9881-53b4d48afabd service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.830191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5cd2-8790-4bd9-9881-53b4d48afabd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.839792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc5cd2-8790-4bd9-9881-53b4d48afabd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.841272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc5cd2-8790-4bd9-9881-53b4d48afabd service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:00.852549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1215/2406] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:00.873521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98a52c-80cc-4878-8ec8-ca404643f90e None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:00.940478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98a52c-80cc-4878-8ec8-ca404643f90e None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:00.945161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:00.946540 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:00.949101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:00.949101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] PATH_INFO: `/v3/users/d633030694cb49a4a478f1f7a6d21c2b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:00.949101 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=d633030694cb49a4a478f1f7a6d21c2b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:00.950405 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.111213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:01.117724 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd633030694cb49a4a478f1f7a6d21c2b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:01.244638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd633030694cb49a4a478f1f7a6d21c2b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:01.245568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd633030694cb49a4a478f1f7a6d21c2b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:01.254114 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd633030694cb49a4a478f1f7a6d21c2b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:01.255218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd633030694cb49a4a478f1f7a6d21c2b'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:01.276651 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1192/2407] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:00 2022] DELETE /identity/v3/users/d633030694cb49a4a478f1f7a6d21c2b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:01.278488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefadbd9-b8e9-447e-99c2-bbcf42c11abb None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.278686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefadbd9-b8e9-447e-99c2-bbcf42c11abb None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.279005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefadbd9-b8e9-447e-99c2-bbcf42c11abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.296586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fefadbd9-b8e9-447e-99c2-bbcf42c11abb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:01.299426 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fefadbd9-b8e9-447e-99c2-bbcf42c11abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.375488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1216/2408] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:01.687960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.689533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.779555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84693866-a096-44fc-b6f4-3cd1f9a85eda admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:01.787130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:01.787838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.788141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.789813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.805096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:01.807347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:01.821247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1193/2409] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:01.827126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5493a-29e5-4250-9187-081f1ae96376 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:01.828015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5493a-29e5-4250-9187-081f1ae96376 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:01.828386 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5493a-29e5-4250-9187-081f1ae96376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:01.849965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-56e5493a-29e5-4250-9187-081f1ae96376 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:01.851224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-56e5493a-29e5-4250-9187-081f1ae96376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:01.922002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1217/2410] 10.222.0.40 () {58 vars in 1009 bytes} [Mon Jul 25 06:06:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:01.934171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:01.935583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.024962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb07c6-c042-44c0-ac36-f280ba6aa56a service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:02.032222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef15eb-95e1-499c-be9b-2d2fcbf110c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:02.032997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef15eb-95e1-499c-be9b-2d2fcbf110c8 service swift] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.033492 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef15eb-95e1-499c-be9b-2d2fcbf110c8 service swift] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.033841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef15eb-95e1-499c-be9b-2d2fcbf110c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.043838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef15eb-95e1-499c-be9b-2d2fcbf110c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:02.045572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef15eb-95e1-499c-be9b-2d2fcbf110c8 service swift] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:02.054186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1194/2411] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:06:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:06:02.502802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5493a-29e5-4250-9187-081f1ae96376 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:02.504841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-56e5493a-29e5-4250-9187-081f1ae96376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.551264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5493a-29e5-4250-9187-081f1ae96376 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:02.561043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:02.562375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.562971 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.563391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] PATH_INFO: `/v3/projects/b75327cab649456ca56a425eacbc4377` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.568171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75327cab649456ca56a425eacbc4377)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:02.571755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:02.746238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75327cab649456ca56a425eacbc4377', 'request_id': 'req-e903074e-44c0-40a1-90f0-8ca5d8468970'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.828448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75327cab649456ca56a425eacbc4377', 'request_id': 'req-e903074e-44c0-40a1-90f0-8ca5d8468970'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.837105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75327cab649456ca56a425eacbc4377', 'request_id': 'req-e903074e-44c0-40a1-90f0-8ca5d8468970'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.870527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] The token cache is being invalidate because project b75327cab649456ca56a425eacbc4377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:02.870942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.871709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e903074e-44c0-40a1-90f0-8ca5d8468970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:02.873508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1218/2412] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:02 2022] DELETE /identity/v3/projects/b75327cab649456ca56a425eacbc4377 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:02.878282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259f71-c5a7-433b-950b-7d1f92efba24 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:02.879113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259f71-c5a7-433b-950b-7d1f92efba24 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:02.879543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259f71-c5a7-433b-950b-7d1f92efba24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:02.909156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f4259f71-c5a7-433b-950b-7d1f92efba24 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:02.911366 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f4259f71-c5a7-433b-950b-7d1f92efba24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:02.978369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:03.001949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1195/2413] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:03.003347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485db554-52ba-408c-8eb1-d615ac3b617b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.003802 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485db554-52ba-408c-8eb1-d615ac3b617b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.004470 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485db554-52ba-408c-8eb1-d615ac3b617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.042421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-485db554-52ba-408c-8eb1-d615ac3b617b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:03.043501 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-485db554-52ba-408c-8eb1-d615ac3b617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.058599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1219/2414] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:03.064409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4259f71-c5a7-433b-950b-7d1f92efba24 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.078168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4259f71-c5a7-433b-950b-7d1f92efba24 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.085544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.088282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.090024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.092522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.093472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:03.096608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.098328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.112051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1196/2415] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:06:03.117503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485db554-52ba-408c-8eb1-d615ac3b617b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.143208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485db554-52ba-408c-8eb1-d615ac3b617b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.148447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3e629-0153-405e-b6d9-8bf3947afb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.149778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3e629-0153-405e-b6d9-8bf3947afb1d admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.150121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3e629-0153-405e-b6d9-8bf3947afb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.150461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f3e629-0153-405e-b6d9-8bf3947afb1d admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.153285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3e629-0153-405e-b6d9-8bf3947afb1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.153285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f3e629-0153-405e-b6d9-8bf3947afb1d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.161871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:03.256810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1220/2416] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:03 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:03.261873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.271798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b0eac-763f-41c3-89ae-b05b03e78485 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.329450 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da17e-05dd-407f-81fb-82eb4a85bdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.330129 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da17e-05dd-407f-81fb-82eb4a85bdfe admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.330415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da17e-05dd-407f-81fb-82eb4a85bdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.330746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318da17e-05dd-407f-81fb-82eb4a85bdfe admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.331491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318da17e-05dd-407f-81fb-82eb4a85bdfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.333472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318da17e-05dd-407f-81fb-82eb4a85bdfe admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.407026 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1197/2417] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:03 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:03.410612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3e629-0153-405e-b6d9-8bf3947afb1d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.421336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f3e629-0153-405e-b6d9-8bf3947afb1d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.426067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e71317-7ea5-4024-859a-ab2f27182fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.426770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e71317-7ea5-4024-859a-ab2f27182fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.427054 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e71317-7ea5-4024-859a-ab2f27182fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.427469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e71317-7ea5-4024-859a-ab2f27182fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.428241 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e71317-7ea5-4024-859a-ab2f27182fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.430360 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e71317-7ea5-4024-859a-ab2f27182fc5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.440980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1221/2418] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:03.446586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da17e-05dd-407f-81fb-82eb4a85bdfe admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.455697 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318da17e-05dd-407f-81fb-82eb4a85bdfe admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.460615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cec03-f262-43c1-964a-f15ae1fedcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.461341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cec03-f262-43c1-964a-f15ae1fedcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.461623 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cec03-f262-43c1-964a-f15ae1fedcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.461897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cec03-f262-43c1-964a-f15ae1fedcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.462624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cec03-f262-43c1-964a-f15ae1fedcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.464251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cec03-f262-43c1-964a-f15ae1fedcf2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.474209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1198/2419] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:03.478399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e71317-7ea5-4024-859a-ab2f27182fc5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.488440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e71317-7ea5-4024-859a-ab2f27182fc5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.493604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e9e94-8dd7-41a8-afa1-eb7e32f234a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.494475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9e94-8dd7-41a8-afa1-eb7e32f234a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.494761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9e94-8dd7-41a8-afa1-eb7e32f234a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.495042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e9e94-8dd7-41a8-afa1-eb7e32f234a5 admin admin] PATH_INFO: `/v3/projects/c665769bcfe34dd09766452e0cd5e273/users/cdbdfc3c348541f1b0f0b95b08cf7665/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.509937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e9e94-8dd7-41a8-afa1-eb7e32f234a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c665769bcfe34dd09766452e0cd5e273, user_id=cdbdfc3c348541f1b0f0b95b08cf7665, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.512702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e9e94-8dd7-41a8-afa1-eb7e32f234a5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.611246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1222/2420] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:03 2022] PUT /identity/v3/projects/c665769bcfe34dd09766452e0cd5e273/users/cdbdfc3c348541f1b0f0b95b08cf7665/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:03.675713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e682f3-0308-4610-b12b-ce5b082af2fb None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.676000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e682f3-0308-4610-b12b-ce5b082af2fb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.676311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e682f3-0308-4610-b12b-ce5b082af2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.702818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-94e682f3-0308-4610-b12b-ce5b082af2fb None None] MFA Rules not processed for user `cdbdfc3c348541f1b0f0b95b08cf7665`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:03.704013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-94e682f3-0308-4610-b12b-ce5b082af2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:03.766415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1199/2421] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:03.770897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e9e94-8dd7-41a8-afa1-eb7e32f234a5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.831092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e9e94-8dd7-41a8-afa1-eb7e32f234a5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.835845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9a13b-0b45-4b3b-aa33-ff4f9224d390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.836939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a13b-0b45-4b3b-aa33-ff4f9224d390 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.837227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a13b-0b45-4b3b-aa33-ff4f9224d390 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.837506 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9a13b-0b45-4b3b-aa33-ff4f9224d390 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.838195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9a13b-0b45-4b3b-aa33-ff4f9224d390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.840149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9a13b-0b45-4b3b-aa33-ff4f9224d390 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:03.921332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1223/2422] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:03 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:03.925432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e682f3-0308-4610-b12b-ce5b082af2fb None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:03.937034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e682f3-0308-4610-b12b-ce5b082af2fb None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:03.978453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:03.993384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c9772-2b7d-44a0-8d17-08e1ff2c7070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:03.994054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9772-2b7d-44a0-8d17-08e1ff2c7070 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:03.994344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9772-2b7d-44a0-8d17-08e1ff2c7070 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:03.994625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c9772-2b7d-44a0-8d17-08e1ff2c7070 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:03.995318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c9772-2b7d-44a0-8d17-08e1ff2c7070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:03.997295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5c9772-2b7d-44a0-8d17-08e1ff2c7070 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.158517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1200/2423] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:03 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:04.162736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9a13b-0b45-4b3b-aa33-ff4f9224d390 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.174563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9a13b-0b45-4b3b-aa33-ff4f9224d390 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.179582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd73bc-f7d9-48e0-b2f2-3928c4022c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.180638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd73bc-f7d9-48e0-b2f2-3928c4022c53 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.181019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd73bc-f7d9-48e0-b2f2-3928c4022c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.181424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd73bc-f7d9-48e0-b2f2-3928c4022c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.182314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd73bc-f7d9-48e0-b2f2-3928c4022c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.184754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bd73bc-f7d9-48e0-b2f2-3928c4022c53 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.203596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c9772-2b7d-44a0-8d17-08e1ff2c7070 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.204727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1224/2424] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:04.218570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5c9772-2b7d-44a0-8d17-08e1ff2c7070 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.224680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096dbad-379a-4273-ab41-694a8d203663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.225373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096dbad-379a-4273-ab41-694a8d203663 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.226160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096dbad-379a-4273-ab41-694a8d203663 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.226160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096dbad-379a-4273-ab41-694a8d203663 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.227901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096dbad-379a-4273-ab41-694a8d203663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.229862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096dbad-379a-4273-ab41-694a8d203663 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.241290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1201/2425] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:04.245963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd73bc-f7d9-48e0-b2f2-3928c4022c53 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.262945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd73bc-f7d9-48e0-b2f2-3928c4022c53 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.267671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c43f1-568d-4df8-82ec-6928a1e542ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.268609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c43f1-568d-4df8-82ec-6928a1e542ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.268895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c43f1-568d-4df8-82ec-6928a1e542ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.269176 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c43f1-568d-4df8-82ec-6928a1e542ca admin admin] PATH_INFO: `/v3/projects/c88f366345e84881b24f27b1673a06f0/users/55691e512fd6468780961c0c4c437d0a/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.284813 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c43f1-568d-4df8-82ec-6928a1e542ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88f366345e84881b24f27b1673a06f0, user_id=55691e512fd6468780961c0c4c437d0a, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.287595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c43f1-568d-4df8-82ec-6928a1e542ca admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.360511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1225/2426] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:04 2022] PUT /identity/v3/projects/c88f366345e84881b24f27b1673a06f0/users/55691e512fd6468780961c0c4c437d0a/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:04.371392 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbabc9-b502-4af5-bd62-68ecda0406b4 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.371686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbabc9-b502-4af5-bd62-68ecda0406b4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.371966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebbabc9-b502-4af5-bd62-68ecda0406b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.399938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-2ebbabc9-b502-4af5-bd62-68ecda0406b4 None None] MFA Rules not processed for user `55691e512fd6468780961c0c4c437d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:04.401163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2ebbabc9-b502-4af5-bd62-68ecda0406b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:04.463081 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1202/2427] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:04.467467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c43f1-568d-4df8-82ec-6928a1e542ca admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.527999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c43f1-568d-4df8-82ec-6928a1e542ca admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.532737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92261c59-dfee-4ab8-8306-c1e80bbbb758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.533396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261c59-dfee-4ab8-8306-c1e80bbbb758 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.533681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261c59-dfee-4ab8-8306-c1e80bbbb758 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.533960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261c59-dfee-4ab8-8306-c1e80bbbb758 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.534659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92261c59-dfee-4ab8-8306-c1e80bbbb758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.536619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92261c59-dfee-4ab8-8306-c1e80bbbb758 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.544739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:04.619042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1226/2428] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:04 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:04.623246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbabc9-b502-4af5-bd62-68ecda0406b4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.635098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebbabc9-b502-4af5-bd62-68ecda0406b4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.718288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a0b1e-57f1-44dc-9b8d-75954a19fec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.722728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a0b1e-57f1-44dc-9b8d-75954a19fec4 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.722728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a0b1e-57f1-44dc-9b8d-75954a19fec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.722728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a0b1e-57f1-44dc-9b8d-75954a19fec4 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.722728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a0b1e-57f1-44dc-9b8d-75954a19fec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.722728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a0b1e-57f1-44dc-9b8d-75954a19fec4 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.818379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1203/2429] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:04 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:04.823111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92261c59-dfee-4ab8-8306-c1e80bbbb758 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.833302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92261c59-dfee-4ab8-8306-c1e80bbbb758 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.838271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d157fc0-ac23-4545-a9ff-fedea867f70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.838932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d157fc0-ac23-4545-a9ff-fedea867f70d admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.839211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d157fc0-ac23-4545-a9ff-fedea867f70d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.839491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d157fc0-ac23-4545-a9ff-fedea867f70d admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.840244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d157fc0-ac23-4545-a9ff-fedea867f70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.841837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d157fc0-ac23-4545-a9ff-fedea867f70d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.855833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1227/2430] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:04.857120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a0b1e-57f1-44dc-9b8d-75954a19fec4 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.868666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a0b1e-57f1-44dc-9b8d-75954a19fec4 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:04.873518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f708c-eb37-4d08-8894-05a362d548df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:04.874445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f708c-eb37-4d08-8894-05a362d548df admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:04.874722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f708c-eb37-4d08-8894-05a362d548df admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:04.874997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f708c-eb37-4d08-8894-05a362d548df admin admin] PATH_INFO: `/v3/projects/71a109bf164d4493a74c7d28c7d9bb5a/users/0231d53cbe85407fac8f89cd68735ddc/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:04.897881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f708c-eb37-4d08-8894-05a362d548df admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a109bf164d4493a74c7d28c7d9bb5a, user_id=0231d53cbe85407fac8f89cd68735ddc, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:04.901468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f708c-eb37-4d08-8894-05a362d548df admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:04.963636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1204/2431] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:04 2022] PUT /identity/v3/projects/71a109bf164d4493a74c7d28c7d9bb5a/users/0231d53cbe85407fac8f89cd68735ddc/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:04.964905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d157fc0-ac23-4545-a9ff-fedea867f70d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:04.975746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d157fc0-ac23-4545-a9ff-fedea867f70d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.038320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec7dbc4-3ecd-4281-a8fb-030e63aa0133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.038993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7dbc4-3ecd-4281-a8fb-030e63aa0133 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.039359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7dbc4-3ecd-4281-a8fb-030e63aa0133 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.039625 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec7dbc4-3ecd-4281-a8fb-030e63aa0133 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.040578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7dbc4-3ecd-4281-a8fb-030e63aa0133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.042180 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec7dbc4-3ecd-4281-a8fb-030e63aa0133 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.057687 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1228/2432] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:05.062015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f708c-eb37-4d08-8894-05a362d548df admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.081063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f708c-eb37-4d08-8894-05a362d548df admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.086691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.087693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.088199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.088565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] PATH_INFO: `/v3/projects/71a109bf164d4493a74c7d28c7d9bb5a/users/0231d53cbe85407fac8f89cd68735ddc/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.090705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a109bf164d4493a74c7d28c7d9bb5a, user_id=0231d53cbe85407fac8f89cd68735ddc, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.093456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.104821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1205/2433] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:05 2022] PUT /identity/v3/projects/71a109bf164d4493a74c7d28c7d9bb5a/users/0231d53cbe85407fac8f89cd68735ddc/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:05.116359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417091-8661-4302-adc0-9ae5570731b9 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.116649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417091-8661-4302-adc0-9ae5570731b9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.116930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de417091-8661-4302-adc0-9ae5570731b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.151584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-de417091-8661-4302-adc0-9ae5570731b9 None None] MFA Rules not processed for user `0231d53cbe85407fac8f89cd68735ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:05.153259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-de417091-8661-4302-adc0-9ae5570731b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.232806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1229/2434] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:05.245945 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:05.247819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:05.261213 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:05.351172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1645a17-122c-4b50-8d45-577a4765a3ca admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:05.355557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297094d-dfe8-4ac6-8d9d-92679849f533 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:05.356271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297094d-dfe8-4ac6-8d9d-92679849f533 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:05.356561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297094d-dfe8-4ac6-8d9d-92679849f533 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:05.356842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297094d-dfe8-4ac6-8d9d-92679849f533 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:05.438462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297094d-dfe8-4ac6-8d9d-92679849f533 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:05.439950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297094d-dfe8-4ac6-8d9d-92679849f533 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:05.451131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1206/2435] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:07.621968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de417091-8661-4302-adc0-9ae5570731b9 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:07.623824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-de417091-8661-4302-adc0-9ae5570731b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:07.637512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de417091-8661-4302-adc0-9ae5570731b9 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:07.641396 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8311eee-f562-4186-addc-554abe8e8a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:07.642044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8311eee-f562-4186-addc-554abe8e8a77 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:07.642326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8311eee-f562-4186-addc-554abe8e8a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:07.642601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8311eee-f562-4186-addc-554abe8e8a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:07.712783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8311eee-f562-4186-addc-554abe8e8a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:07.714276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8311eee-f562-4186-addc-554abe8e8a77 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:07.718084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:07.725660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1230/2436] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:09.366532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297094d-dfe8-4ac6-8d9d-92679849f533 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:09.382637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297094d-dfe8-4ac6-8d9d-92679849f533 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:09.392393 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4c64c-7b7f-438b-a191-0e8bd2f412eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:09.393066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4c64c-7b7f-438b-a191-0e8bd2f412eb service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:09.393532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4c64c-7b7f-438b-a191-0e8bd2f412eb service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:09.393815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4c64c-7b7f-438b-a191-0e8bd2f412eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:09.409614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4c64c-7b7f-438b-a191-0e8bd2f412eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:09.411252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd4c64c-7b7f-438b-a191-0e8bd2f412eb service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:09.422632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1207/2437] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:14.075289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8311eee-f562-4186-addc-554abe8e8a77 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:14.208431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b39cd5-90aa-471c-8b66-95a9c169450a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:14.208950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b39cd5-90aa-471c-8b66-95a9c169450a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:14.209192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b39cd5-90aa-471c-8b66-95a9c169450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:14.215320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8311eee-f562-4186-addc-554abe8e8a77 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:14.228134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:14.229039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:14.229329 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:14.231649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] PATH_INFO: `/v3/users/cdbdfc3c348541f1b0f0b95b08cf7665` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:14.233984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdbdfc3c348541f1b0f0b95b08cf7665)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:14.237911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:14.244032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-79b39cd5-90aa-471c-8b66-95a9c169450a None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:14.246158 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-79b39cd5-90aa-471c-8b66-95a9c169450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:14.322936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1208/2438] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 25 06:06:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:14.335537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b39cd5-90aa-471c-8b66-95a9c169450a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:14.337976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-79b39cd5-90aa-471c-8b66-95a9c169450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:14.400545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:14.420273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b39cd5-90aa-471c-8b66-95a9c169450a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:14.425404 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90149b9-0453-41ab-90c0-b01fb89a7a1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:14.426058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90149b9-0453-41ab-90c0-b01fb89a7a1a service swift] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:14.426345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90149b9-0453-41ab-90c0-b01fb89a7a1a service swift] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:14.426623 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90149b9-0453-41ab-90c0-b01fb89a7a1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:14.437831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90149b9-0453-41ab-90c0-b01fb89a7a1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:14.439607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90149b9-0453-41ab-90c0-b01fb89a7a1a service swift] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:14.451265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1209/2439] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:06:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 25 06:06:14.455999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdbdfc3c348541f1b0f0b95b08cf7665'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:14.546385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdbdfc3c348541f1b0f0b95b08cf7665'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:14.547015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdbdfc3c348541f1b0f0b95b08cf7665'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:14.555531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdbdfc3c348541f1b0f0b95b08cf7665'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:14.556558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdbdfc3c348541f1b0f0b95b08cf7665'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:14.575755 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1231/2440] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:14 2022] DELETE /identity/v3/users/cdbdfc3c348541f1b0f0b95b08cf7665 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:14.580510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066ee02-fce1-4043-b6b0-63ce114b2d10 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:14.580881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066ee02-fce1-4043-b6b0-63ce114b2d10 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:14.581239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066ee02-fce1-4043-b6b0-63ce114b2d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:14.600075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7066ee02-fce1-4043-b6b0-63ce114b2d10 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:14.602798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7066ee02-fce1-4043-b6b0-63ce114b2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:14.678956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1210/2441] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:14.689548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:14.691011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:14.750265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e6a4a-82e7-410d-836a-7ca55f3d048e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:14.761158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07eb8d-ff99-4d49-9067-a1e168d02f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:14.761827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07eb8d-ff99-4d49-9067-a1e168d02f1c service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:14.763706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07eb8d-ff99-4d49-9067-a1e168d02f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:14.763706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07eb8d-ff99-4d49-9067-a1e168d02f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:14.779167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07eb8d-ff99-4d49-9067-a1e168d02f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:14.780794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07eb8d-ff99-4d49-9067-a1e168d02f1c service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:14.792406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1232/2442] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:15.286731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066ee02-fce1-4043-b6b0-63ce114b2d10 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:15.290093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7066ee02-fce1-4043-b6b0-63ce114b2d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:15.307867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066ee02-fce1-4043-b6b0-63ce114b2d10 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:15.316337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:15.317154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:15.317472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:15.317712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] PATH_INFO: `/v3/projects/c665769bcfe34dd09766452e0cd5e273` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:15.319598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c665769bcfe34dd09766452e0cd5e273)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:15.322918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:15.468029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c665769bcfe34dd09766452e0cd5e273', 'request_id': 'req-c60a6fd1-154c-466a-a62d-170fe923901d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.566033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c665769bcfe34dd09766452e0cd5e273', 'request_id': 'req-c60a6fd1-154c-466a-a62d-170fe923901d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.575121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c665769bcfe34dd09766452e0cd5e273', 'request_id': 'req-c60a6fd1-154c-466a-a62d-170fe923901d'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.589986 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] The token cache is being invalidate because project c665769bcfe34dd09766452e0cd5e273 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:15.590425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.591194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.596011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1211/2443] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:15 2022] DELETE /identity/v3/projects/c665769bcfe34dd09766452e0cd5e273 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:15.599924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07eb8d-ff99-4d49-9067-a1e168d02f1c service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:15.611031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07eb8d-ff99-4d49-9067-a1e168d02f1c service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:15.631454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:15.681988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:15.682874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:15.683157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:15.683437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] PATH_INFO: `/v3/users/55691e512fd6468780961c0c4c437d0a` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:15.684718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=55691e512fd6468780961c0c4c437d0a)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:15.686629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:15.881359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55691e512fd6468780961c0c4c437d0a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.977822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55691e512fd6468780961c0c4c437d0a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.978606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55691e512fd6468780961c0c4c437d0a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.986131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55691e512fd6468780961c0c4c437d0a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:15.988937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55691e512fd6468780961c0c4c437d0a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:16.010075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1233/2444] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:15 2022] DELETE /identity/v3/users/55691e512fd6468780961c0c4c437d0a => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:16.634338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:16.636263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:16.654539 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a6fd1-154c-466a-a62d-170fe923901d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:16.755077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:16.755077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:16.755077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:16.755077 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] PATH_INFO: `/v3/projects/c88f366345e84881b24f27b1673a06f0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:16.755779 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88f366345e84881b24f27b1673a06f0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:16.768575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:16.776431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:16.912345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88f366345e84881b24f27b1673a06f0', 'request_id': 'req-885bb95d-8e5d-48be-820b-ee769a5bd1fc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:16.971679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88f366345e84881b24f27b1673a06f0', 'request_id': 'req-885bb95d-8e5d-48be-820b-ee769a5bd1fc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:16.979845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88f366345e84881b24f27b1673a06f0', 'request_id': 'req-885bb95d-8e5d-48be-820b-ee769a5bd1fc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:16.992905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] The token cache is being invalidate because project c88f366345e84881b24f27b1673a06f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:16.993634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:16.994525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:16.996326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1212/2445] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:16 2022] DELETE /identity/v3/projects/c88f366345e84881b24f27b1673a06f0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:17.000623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:17.003547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:17.017185 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e51e33-88ef-4e99-baa1-580ec15c6a7d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:17.079311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:17.080212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:17.080493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:17.080763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] PATH_INFO: `/v3/users/0231d53cbe85407fac8f89cd68735ddc` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:17.081940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0231d53cbe85407fac8f89cd68735ddc)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:17.083852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:17.327439 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:17.335717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0231d53cbe85407fac8f89cd68735ddc'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:17.457406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0231d53cbe85407fac8f89cd68735ddc'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:17.458131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0231d53cbe85407fac8f89cd68735ddc'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:17.466747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0231d53cbe85407fac8f89cd68735ddc'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:17.467593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-db4ae9e1-efb3-4698-ae6a-0cc0aa6f3d38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0231d53cbe85407fac8f89cd68735ddc'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:17.513066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1234/2446] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:16 2022] DELETE /identity/v3/users/0231d53cbe85407fac8f89cd68735ddc => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:18.069278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:18.071377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:18.088861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885bb95d-8e5d-48be-820b-ee769a5bd1fc admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:18.147264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:18.148224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:18.148619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:18.149248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] PATH_INFO: `/v3/projects/71a109bf164d4493a74c7d28c7d9bb5a` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:18.151923 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a109bf164d4493a74c7d28c7d9bb5a)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:18.152400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:18.317782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a109bf164d4493a74c7d28c7d9bb5a', 'request_id': 'req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:18.418708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a109bf164d4493a74c7d28c7d9bb5a', 'request_id': 'req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:18.427022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a109bf164d4493a74c7d28c7d9bb5a', 'request_id': 'req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:18.442893 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] The token cache is being invalidate because project 71a109bf164d4493a74c7d28c7d9bb5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:18.443415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:18.444107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c71ddd1-7c50-4b39-8b8e-544e31fda75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:18.445844 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1213/2447] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:18 2022] DELETE /identity/v3/projects/71a109bf164d4493a74c7d28c7d9bb5a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:18.451197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9d95-62e5-4699-8df9-5086c4320966 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:18.452021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9d95-62e5-4699-8df9-5086c4320966 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:18.452500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd9d95-62e5-4699-8df9-5086c4320966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:18.479863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-3edd9d95-62e5-4699-8df9-5086c4320966 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:18.481394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3edd9d95-62e5-4699-8df9-5086c4320966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:18.564106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1235/2448] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:18.570496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5732-8841-408c-964b-eac6272d7783 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:18.570861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5732-8841-408c-964b-eac6272d7783 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:18.571157 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5732-8841-408c-964b-eac6272d7783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:18.591768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-6b0b5732-8841-408c-964b-eac6272d7783 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:18.593111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-6b0b5732-8841-408c-964b-eac6272d7783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:18.602781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1214/2449] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:18.612379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9d95-62e5-4699-8df9-5086c4320966 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:18.623095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd9d95-62e5-4699-8df9-5086c4320966 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:18.628051 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:18.628766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:18.629046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:18.629319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:18.630236 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:18.630552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:18.632262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:18.646393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1236/2450] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:18.651911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b5732-8841-408c-964b-eac6272d7783 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:18.665286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b5732-8841-408c-964b-eac6272d7783 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:18.674030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68376cb1-1ac6-4f2a-b437-b6f9a48c16fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:18.674742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68376cb1-1ac6-4f2a-b437-b6f9a48c16fe admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:18.675032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68376cb1-1ac6-4f2a-b437-b6f9a48c16fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:18.678209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68376cb1-1ac6-4f2a-b437-b6f9a48c16fe admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:18.678209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68376cb1-1ac6-4f2a-b437-b6f9a48c16fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:18.678209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68376cb1-1ac6-4f2a-b437-b6f9a48c16fe admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:18.695869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:18.763358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1215/2451] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:18 2022] POST /identity/v3/projects => generated 371 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:18.764709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:18.778510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f077e05-4627-4508-86e8-6c7b64449821 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:18.862288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbb459-525a-451e-b82c-9692e6f8cfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:18.863018 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbb459-525a-451e-b82c-9692e6f8cfea admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:18.863300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbb459-525a-451e-b82c-9692e6f8cfea admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:18.863567 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbb459-525a-451e-b82c-9692e6f8cfea admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:18.864293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbb459-525a-451e-b82c-9692e6f8cfea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:18.866222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bbb459-525a-451e-b82c-9692e6f8cfea admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:18.911240 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aae79-fc4f-4a91-a58b-9bd94d5acbf2 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:18.911535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aae79-fc4f-4a91-a58b-9bd94d5acbf2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:18.911819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95aae79-fc4f-4a91-a58b-9bd94d5acbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:18.930991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a95aae79-fc4f-4a91-a58b-9bd94d5acbf2 None None] MFA Rules not processed for user `2cdbdedd9ba34e81916d5104157dfd10`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:18.932253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a95aae79-fc4f-4a91-a58b-9bd94d5acbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:18.987223 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1216/2452] 10.222.0.40 () {58 vars in 967 bytes} [Mon Jul 25 06:06:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:19.002937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1237/2453] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:18 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:19.002937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbb459-525a-451e-b82c-9692e6f8cfea admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:19.003210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aae79-fc4f-4a91-a58b-9bd94d5acbf2 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:19.007966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a95aae79-fc4f-4a91-a58b-9bd94d5acbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:19.016875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bbb459-525a-451e-b82c-9692e6f8cfea admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:19.023151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2922f4-092f-4240-b3cd-eb8ea64b1fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:19.024622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2922f4-092f-4240-b3cd-eb8ea64b1fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:19.024902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2922f4-092f-4240-b3cd-eb8ea64b1fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:19.025424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2922f4-092f-4240-b3cd-eb8ea64b1fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:19.026227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2922f4-092f-4240-b3cd-eb8ea64b1fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:19.027898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2922f4-092f-4240-b3cd-eb8ea64b1fc3 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:19.049884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1238/2454] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:19.054384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2922f4-092f-4240-b3cd-eb8ea64b1fc3 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:19.073424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2922f4-092f-4240-b3cd-eb8ea64b1fc3 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:19.078846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8bdfb-77c3-4990-86ed-602a1090ae37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:19.079732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8bdfb-77c3-4990-86ed-602a1090ae37 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:19.080016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8bdfb-77c3-4990-86ed-602a1090ae37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:19.080343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8bdfb-77c3-4990-86ed-602a1090ae37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:19.081086 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8bdfb-77c3-4990-86ed-602a1090ae37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:19.082977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b8bdfb-77c3-4990-86ed-602a1090ae37 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:19.098922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95aae79-fc4f-4a91-a58b-9bd94d5acbf2 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:19.100932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1239/2455] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:19.103812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8bdfb-77c3-4990-86ed-602a1090ae37 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:19.107837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099338a0-27b1-4419-96ea-21162a8e05ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf71e1ef5acb429b96219888830181d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:19.109509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099338a0-27b1-4419-96ea-21162a8e05ff service swift] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:19.110164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099338a0-27b1-4419-96ea-21162a8e05ff service swift] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:19.110710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099338a0-27b1-4419-96ea-21162a8e05ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:19.115623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8bdfb-77c3-4990-86ed-602a1090ae37 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:19.122044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:19.122646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:19.122927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:19.123200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] PATH_INFO: `/v3/projects/a06728e0b0f6484cae09bcef4becc45d/users/7c406ef1c63f49cabc03d5f2973d3a6e/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:19.129230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099338a0-27b1-4419-96ea-21162a8e05ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:19.130860 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099338a0-27b1-4419-96ea-21162a8e05ff service swift] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:19.138643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:19.144628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a06728e0b0f6484cae09bcef4becc45d, user_id=7c406ef1c63f49cabc03d5f2973d3a6e, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:19.150832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:19.160195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1217/2456] 10.222.0.40 () {60 vars in 1384 bytes} [Mon Jul 25 06:06:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 25 06:06:19.283928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1240/2457] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:19 2022] PUT /identity/v3/projects/a06728e0b0f6484cae09bcef4becc45d/users/7c406ef1c63f49cabc03d5f2973d3a6e/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:19.295204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6e76-6d92-42b4-a51e-606416338c7a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:19.295521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6e76-6d92-42b4-a51e-606416338c7a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:19.295835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f6e76-6d92-42b4-a51e-606416338c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:19.323930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-bd1f6e76-6d92-42b4-a51e-606416338c7a None None] MFA Rules not processed for user `7c406ef1c63f49cabc03d5f2973d3a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:19.325371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bd1f6e76-6d92-42b4-a51e-606416338c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:19.395452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1218/2458] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:19.407917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:19.409648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:19.473028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b4943-5aea-474a-bf8b-6d06641baa83 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:19.477324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c0883bc26a04de2916c78df342d2023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:19.477984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:19.478321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:19.478656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:19.490484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:19.492126 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:19.503998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1241/2459] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:21.355715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6e76-6d92-42b4-a51e-606416338c7a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:21.357186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bd1f6e76-6d92-42b4-a51e-606416338c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:21.379386 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:21.420452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f6e76-6d92-42b4-a51e-606416338c7a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:21.426949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:21.427700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:21.428064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:21.428460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:21.430615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:21.545034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:21.546668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:21.558820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1219/2460] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:25.282988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:25.286837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:25.307700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78834c45-0c99-46bc-8d40-af3b16aaa499 service glance] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:25.319909 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:25.321661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:25.321945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:25.322231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] PATH_INFO: `/v3/users/e13b80595f1d4c8cb4522657c9a88cd6` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:25.323577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e13b80595f1d4c8cb4522657c9a88cd6)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:25.325693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:25.522498 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e13b80595f1d4c8cb4522657c9a88cd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.615458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e13b80595f1d4c8cb4522657c9a88cd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.616089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e13b80595f1d4c8cb4522657c9a88cd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.628299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e13b80595f1d4c8cb4522657c9a88cd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.628299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e13b80595f1d4c8cb4522657c9a88cd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:25.656507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1242/2461] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:25 2022] DELETE /identity/v3/users/e13b80595f1d4c8cb4522657c9a88cd6 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:26.017489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.041900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190cb932-ac0b-420e-8457-65c21626eb59 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:26.129093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:26.129970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.130453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.130642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] PATH_INFO: `/v3/projects/a18a468edefd401c835e7e368ac740e5` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.131971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18a468edefd401c835e7e368ac740e5)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:26.133970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:26.270021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18a468edefd401c835e7e368ac740e5', 'request_id': 'req-98ce5830-206f-4995-9eba-ae97fe4e07d4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.398380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18a468edefd401c835e7e368ac740e5', 'request_id': 'req-98ce5830-206f-4995-9eba-ae97fe4e07d4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.407926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18a468edefd401c835e7e368ac740e5', 'request_id': 'req-98ce5830-206f-4995-9eba-ae97fe4e07d4'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.414798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.416544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:26.417363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:26.421890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] The token cache is being invalidate because project a18a468edefd401c835e7e368ac740e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:26.422203 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.422889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.425232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1220/2462] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:26 2022] DELETE /identity/v3/projects/a18a468edefd401c835e7e368ac740e5 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:26.428699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:26.431144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:26.433546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faca649-6657-487b-b662-0c0e7b6a5c53 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:26.500691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:26.501406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.501642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.501923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] PATH_INFO: `/v3/users/a7224418acee432182bf3195bfc8fd99` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.503285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7224418acee432182bf3195bfc8fd99)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:26.505302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:26.513339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:26.520881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5830-206f-4995-9eba-ae97fe4e07d4 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:26.530700 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:26.531825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:26.532716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:26.533225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] PATH_INFO: `/v3/users/9a8f3d909c464a08bc398ec9cbd64e3e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:26.535685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8f3d909c464a08bc398ec9cbd64e3e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:26.538458 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:26.692036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7224418acee432182bf3195bfc8fd99'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.693237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8f3d909c464a08bc398ec9cbd64e3e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.771956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7224418acee432182bf3195bfc8fd99'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.772616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7224418acee432182bf3195bfc8fd99'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.782097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7224418acee432182bf3195bfc8fd99'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.782848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7224418acee432182bf3195bfc8fd99'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.801569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1243/2463] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:26 2022] DELETE /identity/v3/users/a7224418acee432182bf3195bfc8fd99 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:26.832686 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8f3d909c464a08bc398ec9cbd64e3e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.833335 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8f3d909c464a08bc398ec9cbd64e3e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.842162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8f3d909c464a08bc398ec9cbd64e3e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.848485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8f3d909c464a08bc398ec9cbd64e3e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:26.868753 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1221/2464] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:26 2022] DELETE /identity/v3/users/9a8f3d909c464a08bc398ec9cbd64e3e => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:27.092401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.093961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.164781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9af2e6-93d3-43bd-865b-815adc95cccf admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.174509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.176481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.239186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.240466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.240750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.241146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] PATH_INFO: `/v3/projects/5ea60662f93343019fe380b6e13d5d18` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.243736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ea60662f93343019fe380b6e13d5d18)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.246152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.256072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e869403-8c4c-4b78-9e06-3c8e3dadd151 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.261439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.262315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.262614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.262900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] PATH_INFO: `/v3/projects/2c590ee21b9f407aba4b79240f33394a` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.264268 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c590ee21b9f407aba4b79240f33394a)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.266183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.289768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:27.350731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:27.362783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ea60662f93343019fe380b6e13d5d18', 'request_id': 'req-2d4da97f-e13a-4193-85f1-3a725efcb85a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.393160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c590ee21b9f407aba4b79240f33394a', 'request_id': 'req-20202ce6-dea4-46ca-a63c-26677b6a2b80'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.429133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ea60662f93343019fe380b6e13d5d18', 'request_id': 'req-2d4da97f-e13a-4193-85f1-3a725efcb85a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.435981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ea60662f93343019fe380b6e13d5d18', 'request_id': 'req-2d4da97f-e13a-4193-85f1-3a725efcb85a'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.448036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] The token cache is being invalidate because project 5ea60662f93343019fe380b6e13d5d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:27.448428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.449088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2d4da97f-e13a-4193-85f1-3a725efcb85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.450631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1244/2465] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:27 2022] DELETE /identity/v3/projects/5ea60662f93343019fe380b6e13d5d18 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:27.456139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda24dca-98db-452d-b569-593738bb8ff4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.456139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda24dca-98db-452d-b569-593738bb8ff4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.456709 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda24dca-98db-452d-b569-593738bb8ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.482512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fda24dca-98db-452d-b569-593738bb8ff4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:27.483727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fda24dca-98db-452d-b569-593738bb8ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.486087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c590ee21b9f407aba4b79240f33394a', 'request_id': 'req-20202ce6-dea4-46ca-a63c-26677b6a2b80'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.493036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c590ee21b9f407aba4b79240f33394a', 'request_id': 'req-20202ce6-dea4-46ca-a63c-26677b6a2b80'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.515257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] The token cache is being invalidate because project 2c590ee21b9f407aba4b79240f33394a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:27.515557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.516237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20202ce6-dea4-46ca-a63c-26677b6a2b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:27.517794 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1222/2466] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:27 2022] DELETE /identity/v3/projects/2c590ee21b9f407aba4b79240f33394a => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:27.551622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1245/2467] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:27.560470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d991ea-8643-4135-a458-6acfc246816c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.560843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d991ea-8643-4135-a458-6acfc246816c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.561062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d991ea-8643-4135-a458-6acfc246816c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.581267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-97d991ea-8643-4135-a458-6acfc246816c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:27.582600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-97d991ea-8643-4135-a458-6acfc246816c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:27.591781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1223/2468] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:27.596425 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda24dca-98db-452d-b569-593738bb8ff4 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.608105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda24dca-98db-452d-b569-593738bb8ff4 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.612806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.613491 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.613773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.614044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.614788 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:27.615104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.616767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.635905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1246/2469] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:06:27.639016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d991ea-8643-4135-a458-6acfc246816c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.649743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d991ea-8643-4135-a458-6acfc246816c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.654649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c0c5-7f87-45fa-a50f-7fb8f0299bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.655330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c0c5-7f87-45fa-a50f-7fb8f0299bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.655622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c0c5-7f87-45fa-a50f-7fb8f0299bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.655900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1c0c5-7f87-45fa-a50f-7fb8f0299bf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.656640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1c0c5-7f87-45fa-a50f-7fb8f0299bf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.658591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1c0c5-7f87-45fa-a50f-7fb8f0299bf3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.726137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1224/2470] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/projects => generated 355 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:27.728446 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.744597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbf9ba-cfd2-4f06-8339-b7e06de0334e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.813321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24916b-c3e9-4d3c-9ff8-31a4b8095dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.814138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24916b-c3e9-4d3c-9ff8-31a4b8095dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.814423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24916b-c3e9-4d3c-9ff8-31a4b8095dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.814843 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24916b-c3e9-4d3c-9ff8-31a4b8095dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.815643 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24916b-c3e9-4d3c-9ff8-31a4b8095dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.817729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24916b-c3e9-4d3c-9ff8-31a4b8095dc1 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.907333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1247/2471] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:27 2022] POST /identity/v3/users => generated 278 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:27.908645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c0c5-7f87-45fa-a50f-7fb8f0299bf3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.920037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1c0c5-7f87-45fa-a50f-7fb8f0299bf3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.929553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f275c-cc97-4482-b042-a740105a61d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.930960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f275c-cc97-4482-b042-a740105a61d1 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.931251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f275c-cc97-4482-b042-a740105a61d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.931581 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f275c-cc97-4482-b042-a740105a61d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.932654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f275c-cc97-4482-b042-a740105a61d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.934364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f275c-cc97-4482-b042-a740105a61d1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.945826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1225/2472] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:27.950086 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24916b-c3e9-4d3c-9ff8-31a4b8095dc1 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.960074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24916b-c3e9-4d3c-9ff8-31a4b8095dc1 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.965368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264317a0-e80b-4671-887f-bc1c43ae158a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:27.966061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264317a0-e80b-4671-887f-bc1c43ae158a admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:27.966347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264317a0-e80b-4671-887f-bc1c43ae158a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:27.966642 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264317a0-e80b-4671-887f-bc1c43ae158a admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:27.967401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264317a0-e80b-4671-887f-bc1c43ae158a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:27.969103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264317a0-e80b-4671-887f-bc1c43ae158a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:27.983532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1248/2473] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:27.985079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f275c-cc97-4482-b042-a740105a61d1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:27.995194 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f275c-cc97-4482-b042-a740105a61d1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:27.999783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54df4e4-960c-4fc4-930e-a5f23f8c15d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.000713 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df4e4-960c-4fc4-930e-a5f23f8c15d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.001014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df4e4-960c-4fc4-930e-a5f23f8c15d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.001272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df4e4-960c-4fc4-930e-a5f23f8c15d3 admin admin] PATH_INFO: `/v3/projects/ab50d48ef49041c08031ba837a13f624/users/c60d71f78d574be5a2ae6f925fd3bf65/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.016873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54df4e4-960c-4fc4-930e-a5f23f8c15d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab50d48ef49041c08031ba837a13f624, user_id=c60d71f78d574be5a2ae6f925fd3bf65, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.019544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54df4e4-960c-4fc4-930e-a5f23f8c15d3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.116014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1226/2474] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:27 2022] PUT /identity/v3/projects/ab50d48ef49041c08031ba837a13f624/users/c60d71f78d574be5a2ae6f925fd3bf65/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:28.126748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a6c9f-a8af-4946-8c44-1d37d7dcec27 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.127070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a6c9f-a8af-4946-8c44-1d37d7dcec27 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.127357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a6c9f-a8af-4946-8c44-1d37d7dcec27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.162037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-181a6c9f-a8af-4946-8c44-1d37d7dcec27 None None] MFA Rules not processed for user `c60d71f78d574be5a2ae6f925fd3bf65`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:28.163279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-181a6c9f-a8af-4946-8c44-1d37d7dcec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:28.182581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:28.233923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1249/2475] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:28 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:28.238267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54df4e4-960c-4fc4-930e-a5f23f8c15d3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:28.309526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54df4e4-960c-4fc4-930e-a5f23f8c15d3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:28.314347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068391e-23ea-4b54-935d-e6f627049299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.318846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068391e-23ea-4b54-935d-e6f627049299 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.318846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068391e-23ea-4b54-935d-e6f627049299 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.318846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068391e-23ea-4b54-935d-e6f627049299 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.318846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068391e-23ea-4b54-935d-e6f627049299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.318846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068391e-23ea-4b54-935d-e6f627049299 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.404247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1227/2476] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:28 2022] POST /identity/v3/projects => generated 357 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:28.408139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a6c9f-a8af-4946-8c44-1d37d7dcec27 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:28.420258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a6c9f-a8af-4946-8c44-1d37d7dcec27 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:28.483262 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb90113-5800-4664-826e-d4ba57c5dc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.484015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90113-5800-4664-826e-d4ba57c5dc9f admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.484259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90113-5800-4664-826e-d4ba57c5dc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.484543 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90113-5800-4664-826e-d4ba57c5dc9f admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.485251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb90113-5800-4664-826e-d4ba57c5dc9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.487203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb90113-5800-4664-826e-d4ba57c5dc9f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.565941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1250/2477] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:28 2022] POST /identity/v3/users => generated 285 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:28.571677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068391e-23ea-4b54-935d-e6f627049299 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:28.588225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068391e-23ea-4b54-935d-e6f627049299 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:28.598948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173dd1e-345d-4c19-b514-aeaf9be0873f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.599663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173dd1e-345d-4c19-b514-aeaf9be0873f admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.599953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173dd1e-345d-4c19-b514-aeaf9be0873f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.600289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173dd1e-345d-4c19-b514-aeaf9be0873f admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.601046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173dd1e-345d-4c19-b514-aeaf9be0873f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.603426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173dd1e-345d-4c19-b514-aeaf9be0873f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.620238 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1228/2478] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:28.621276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb90113-5800-4664-826e-d4ba57c5dc9f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:28.634140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb90113-5800-4664-826e-d4ba57c5dc9f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:28.646782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c325f-f7c6-4856-bf6d-fffdc4cb3c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.649115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c325f-f7c6-4856-bf6d-fffdc4cb3c58 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.649115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c325f-f7c6-4856-bf6d-fffdc4cb3c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.649115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98c325f-f7c6-4856-bf6d-fffdc4cb3c58 admin admin] PATH_INFO: `/v3/projects/51cad71a48844196955f5e61efa9a2bf/users/f962481419d34327b21cd736b065f313/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.669452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c325f-f7c6-4856-bf6d-fffdc4cb3c58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51cad71a48844196955f5e61efa9a2bf, user_id=f962481419d34327b21cd736b065f313, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.672227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98c325f-f7c6-4856-bf6d-fffdc4cb3c58 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.760441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1251/2479] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:28 2022] PUT /identity/v3/projects/51cad71a48844196955f5e61efa9a2bf/users/f962481419d34327b21cd736b065f313/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:28.765506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173dd1e-345d-4c19-b514-aeaf9be0873f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:28.777633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173dd1e-345d-4c19-b514-aeaf9be0873f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:28.801152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:28.847950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4332c71-6247-4e3f-8d88-7231e9ead09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.848654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4332c71-6247-4e3f-8d88-7231e9ead09a admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.848966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4332c71-6247-4e3f-8d88-7231e9ead09a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.849256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4332c71-6247-4e3f-8d88-7231e9ead09a admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.849962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4332c71-6247-4e3f-8d88-7231e9ead09a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.852645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4332c71-6247-4e3f-8d88-7231e9ead09a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.863584 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1229/2480] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:28.867982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c325f-f7c6-4856-bf6d-fffdc4cb3c58 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:28.881889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98c325f-f7c6-4856-bf6d-fffdc4cb3c58 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:28.887227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:28.888145 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.888428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.888703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] PATH_INFO: `/v3/projects/51cad71a48844196955f5e61efa9a2bf/users/f962481419d34327b21cd736b065f313/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.890762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51cad71a48844196955f5e61efa9a2bf, user_id=f962481419d34327b21cd736b065f313, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:28.893438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:28.904785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1252/2481] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:28 2022] PUT /identity/v3/projects/51cad71a48844196955f5e61efa9a2bf/users/f962481419d34327b21cd736b065f313/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:28.917220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26f76-5328-435a-982c-a6a0d70058c6 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:28.919466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26f76-5328-435a-982c-a6a0d70058c6 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:28.919466 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b26f76-5328-435a-982c-a6a0d70058c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:28.944563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-85b26f76-5328-435a-982c-a6a0d70058c6 None None] MFA Rules not processed for user `f962481419d34327b21cd736b065f313`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:28.946074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-85b26f76-5328-435a-982c-a6a0d70058c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:29.027472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1230/2482] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:28 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:29.035306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:29.037069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:29.165628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d9d2f-60e8-4197-bb06-f2e219a81d46 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:29.169736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:29.172234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.172234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.172234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.184672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:29.186377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:29.204153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1253/2483] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:29 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:29.504741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b26f76-5328-435a-982c-a6a0d70058c6 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:29.514108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b26f76-5328-435a-982c-a6a0d70058c6 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:29.517869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:29.518599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:29.518957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:29.519242 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:29.585334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:29.587350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:29.597262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1231/2484] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:29 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:35.814194 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:35.815601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:35.830111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:35.880733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5e390-7aa6-4de4-a2f4-2b636951bfad service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:35.881747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:35.882398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:35.882710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:35.882981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:35.884927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:35.951298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:35.952690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:35.962648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1254/2485] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:39.592453 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=82 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:39.679032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.680575 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:39.782228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651ec44-8b78-4e18-9baf-0b9439305823 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.787324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.787981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.788311 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.788587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.790472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:39.802410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.805290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:39.815275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1232/2486] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:39.841370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:39.842891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:39.856433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1aeb63-68c9-42f7-96c0-f21977ceea15 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:39.861376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:39.862369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:39.862652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:39.862928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] PATH_INFO: `/v3/users/8eedefd8d94047478663ceab8962d738` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:39.864379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eedefd8d94047478663ceab8962d738)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:39.866289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.060068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eedefd8d94047478663ceab8962d738'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.131672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eedefd8d94047478663ceab8962d738'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.132327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eedefd8d94047478663ceab8962d738'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.140304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eedefd8d94047478663ceab8962d738'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.143777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-62b32067-216a-4738-b250-71ac1b380f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eedefd8d94047478663ceab8962d738'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.160980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1255/2487] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:39 2022] DELETE /identity/v3/users/8eedefd8d94047478663ceab8962d738 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:40.402933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.414275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a637f43-53ad-4210-a7d8-fb07d48f7920 service placement] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.475284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.476288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.476574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.476846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] PATH_INFO: `/v3/projects/a6fd13a3dd1d4e2b83cf6e44160adf35` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.478669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6fd13a3dd1d4e2b83cf6e44160adf35)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.480796 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.565742 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6fd13a3dd1d4e2b83cf6e44160adf35', 'request_id': 'req-61cf3be5-7114-456e-a6d4-29e4bd624bf9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.611232 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6fd13a3dd1d4e2b83cf6e44160adf35', 'request_id': 'req-61cf3be5-7114-456e-a6d4-29e4bd624bf9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.616856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6fd13a3dd1d4e2b83cf6e44160adf35', 'request_id': 'req-61cf3be5-7114-456e-a6d4-29e4bd624bf9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.628959 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] The token cache is being invalidate because project a6fd13a3dd1d4e2b83cf6e44160adf35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:40.629244 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.629915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-61cf3be5-7114-456e-a6d4-29e4bd624bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:40.631459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1233/2488] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:40 2022] DELETE /identity/v3/projects/a6fd13a3dd1d4e2b83cf6e44160adf35 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:40.637050 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191bf9-b2e6-4345-afdc-4089a07bc1a8 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.637354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191bf9-b2e6-4345-afdc-4089a07bc1a8 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.637638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191bf9-b2e6-4345-afdc-4089a07bc1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.655996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b3191bf9-b2e6-4345-afdc-4089a07bc1a8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:40.657162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b3191bf9-b2e6-4345-afdc-4089a07bc1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:40.686496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:40.716940 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1256/2489] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:40.724673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7565f4-a8ef-4064-a7f7-77cbfaa44596 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.724962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7565f4-a8ef-4064-a7f7-77cbfaa44596 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.725337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7565f4-a8ef-4064-a7f7-77cbfaa44596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.744568 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-0b7565f4-a8ef-4064-a7f7-77cbfaa44596 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:40.745777 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-0b7565f4-a8ef-4064-a7f7-77cbfaa44596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:40.753956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1234/2490] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:40.758370 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191bf9-b2e6-4345-afdc-4089a07bc1a8 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.769246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191bf9-b2e6-4345-afdc-4089a07bc1a8 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.773721 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.774657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.776084 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.776357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.777052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:40.777368 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.779016 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.792076 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1257/2491] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:40.796340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7565f4-a8ef-4064-a7f7-77cbfaa44596 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.805720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7565f4-a8ef-4064-a7f7-77cbfaa44596 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.810833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177e46d-34c3-42c2-9d3c-7174ce45ccbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.811530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177e46d-34c3-42c2-9d3c-7174ce45ccbe admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.811869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177e46d-34c3-42c2-9d3c-7174ce45ccbe admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.812181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0177e46d-34c3-42c2-9d3c-7174ce45ccbe admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.812890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177e46d-34c3-42c2-9d3c-7174ce45ccbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.814808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0177e46d-34c3-42c2-9d3c-7174ce45ccbe admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:40.824988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:40.891087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1235/2492] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:40 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:40.895165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:40.906394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3c3b0-c2ee-4598-bc42-7a98634e948c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:40.960980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f34372-990a-4bdd-8c62-84349d7b5cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:40.961650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34372-990a-4bdd-8c62-84349d7b5cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:40.961932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34372-990a-4bdd-8c62-84349d7b5cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:40.962725 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f34372-990a-4bdd-8c62-84349d7b5cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:40.963605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f34372-990a-4bdd-8c62-84349d7b5cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:40.965563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f34372-990a-4bdd-8c62-84349d7b5cc7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.090623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1258/2493] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:40 2022] POST /identity/v3/users => generated 303 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:41.094798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177e46d-34c3-42c2-9d3c-7174ce45ccbe admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.105285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0177e46d-34c3-42c2-9d3c-7174ce45ccbe admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.110225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a5aef-c478-4181-a1c7-ac185b0722fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.110891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a5aef-c478-4181-a1c7-ac185b0722fa admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.111177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a5aef-c478-4181-a1c7-ac185b0722fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.111457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a5aef-c478-4181-a1c7-ac185b0722fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.112288 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a5aef-c478-4181-a1c7-ac185b0722fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.113948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a5aef-c478-4181-a1c7-ac185b0722fa admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.124474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1236/2494] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:41.130597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f34372-990a-4bdd-8c62-84349d7b5cc7 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.141611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f34372-990a-4bdd-8c62-84349d7b5cc7 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.146298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafcdc4-6bca-4910-8d0e-4784d52e723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.146961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafcdc4-6bca-4910-8d0e-4784d52e723e admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.147247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafcdc4-6bca-4910-8d0e-4784d52e723e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.147529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafcdc4-6bca-4910-8d0e-4784d52e723e admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.148291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafcdc4-6bca-4910-8d0e-4784d52e723e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.150997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafcdc4-6bca-4910-8d0e-4784d52e723e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.162907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1259/2495] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:41.167062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a5aef-c478-4181-a1c7-ac185b0722fa admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.183850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a5aef-c478-4181-a1c7-ac185b0722fa admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.183850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132b010-aa9a-46ff-9e86-e7086d2202d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.183850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132b010-aa9a-46ff-9e86-e7086d2202d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.183850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132b010-aa9a-46ff-9e86-e7086d2202d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.184300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132b010-aa9a-46ff-9e86-e7086d2202d0 admin admin] PATH_INFO: `/v3/projects/36cd278953ab46c08b1eb2cea55be8f4/users/02c649fa52ad454f920689a2aed4a020/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.199533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132b010-aa9a-46ff-9e86-e7086d2202d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cd278953ab46c08b1eb2cea55be8f4, user_id=02c649fa52ad454f920689a2aed4a020, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.202231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132b010-aa9a-46ff-9e86-e7086d2202d0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.267094 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1237/2496] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:41 2022] PUT /identity/v3/projects/36cd278953ab46c08b1eb2cea55be8f4/users/02c649fa52ad454f920689a2aed4a020/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:41.279510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bacc-f061-4d6c-a521-168d114ccf26 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.279930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bacc-f061-4d6c-a521-168d114ccf26 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.280333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc1bacc-f061-4d6c-a521-168d114ccf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.306875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0cc1bacc-f061-4d6c-a521-168d114ccf26 None None] MFA Rules not processed for user `02c649fa52ad454f920689a2aed4a020`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:41.308166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0cc1bacc-f061-4d6c-a521-168d114ccf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:41.370139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1260/2497] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:41.374479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132b010-aa9a-46ff-9e86-e7086d2202d0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.434943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132b010-aa9a-46ff-9e86-e7086d2202d0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.439470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116025bc-6393-4f33-8e16-d3db9f817c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.440163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116025bc-6393-4f33-8e16-d3db9f817c5f admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.440452 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116025bc-6393-4f33-8e16-d3db9f817c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.440728 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116025bc-6393-4f33-8e16-d3db9f817c5f admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.441444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116025bc-6393-4f33-8e16-d3db9f817c5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.443330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116025bc-6393-4f33-8e16-d3db9f817c5f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.525121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1238/2498] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:41 2022] POST /identity/v3/projects => generated 403 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:41.529299 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1bacc-f061-4d6c-a521-168d114ccf26 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.540263 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc1bacc-f061-4d6c-a521-168d114ccf26 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.566301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:41.596206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342a66c-96f3-489a-896c-b70e168a4a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.596888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342a66c-96f3-489a-896c-b70e168a4a62 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.597384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342a66c-96f3-489a-896c-b70e168a4a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.597899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342a66c-96f3-489a-896c-b70e168a4a62 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.599034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342a66c-96f3-489a-896c-b70e168a4a62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.600983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342a66c-96f3-489a-896c-b70e168a4a62 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.695205 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1261/2499] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:41 2022] POST /identity/v3/users => generated 308 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:41.699635 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116025bc-6393-4f33-8e16-d3db9f817c5f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.709976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116025bc-6393-4f33-8e16-d3db9f817c5f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.714967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cda3f8-cdf4-4c74-af3b-fd94fd5e9258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.715623 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cda3f8-cdf4-4c74-af3b-fd94fd5e9258 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.715903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cda3f8-cdf4-4c74-af3b-fd94fd5e9258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.716212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cda3f8-cdf4-4c74-af3b-fd94fd5e9258 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.716936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cda3f8-cdf4-4c74-af3b-fd94fd5e9258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.718522 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cda3f8-cdf4-4c74-af3b-fd94fd5e9258 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.729703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1239/2500] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:41.733932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342a66c-96f3-489a-896c-b70e168a4a62 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.744579 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342a66c-96f3-489a-896c-b70e168a4a62 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.749319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef65e-2a07-4417-95e7-36b802aef347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.750208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef65e-2a07-4417-95e7-36b802aef347 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.750493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef65e-2a07-4417-95e7-36b802aef347 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.750768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabef65e-2a07-4417-95e7-36b802aef347 admin admin] PATH_INFO: `/v3/projects/00b52f9c7b09424ab4fba0ee1079642e/users/b8e5457d3d2a423785c69b2da1f61966/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.766578 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabef65e-2a07-4417-95e7-36b802aef347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b52f9c7b09424ab4fba0ee1079642e, user_id=b8e5457d3d2a423785c69b2da1f61966, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.769324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabef65e-2a07-4417-95e7-36b802aef347 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.863134 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1262/2501] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:41 2022] PUT /identity/v3/projects/00b52f9c7b09424ab4fba0ee1079642e/users/b8e5457d3d2a423785c69b2da1f61966/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:41.867215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cda3f8-cdf4-4c74-af3b-fd94fd5e9258 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.878414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cda3f8-cdf4-4c74-af3b-fd94fd5e9258 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.937356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e272c-3cc1-4bcf-b694-cc894d852bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.938027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e272c-3cc1-4bcf-b694-cc894d852bfa admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.938307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e272c-3cc1-4bcf-b694-cc894d852bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.938583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e272c-3cc1-4bcf-b694-cc894d852bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.939301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e272c-3cc1-4bcf-b694-cc894d852bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.940912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e272c-3cc1-4bcf-b694-cc894d852bfa admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.950347 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1240/2502] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:41.954481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef65e-2a07-4417-95e7-36b802aef347 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:41.964838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabef65e-2a07-4417-95e7-36b802aef347 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:41.969374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:41.970247 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:41.970526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:41.970797 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] PATH_INFO: `/v3/projects/00b52f9c7b09424ab4fba0ee1079642e/users/b8e5457d3d2a423785c69b2da1f61966/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:41.972787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b52f9c7b09424ab4fba0ee1079642e, user_id=b8e5457d3d2a423785c69b2da1f61966, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:41.975389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:41.985917 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1263/2503] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:41 2022] PUT /identity/v3/projects/00b52f9c7b09424ab4fba0ee1079642e/users/b8e5457d3d2a423785c69b2da1f61966/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:41.996636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f80ea-f6c2-4af2-8222-bd0eec1e3418 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.002116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f80ea-f6c2-4af2-8222-bd0eec1e3418 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.002116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f80ea-f6c2-4af2-8222-bd0eec1e3418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.023139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ae0f80ea-f6c2-4af2-8222-bd0eec1e3418 None None] MFA Rules not processed for user `b8e5457d3d2a423785c69b2da1f61966`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:42.024688 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ae0f80ea-f6c2-4af2-8222-bd0eec1e3418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.037009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:42.094447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1241/2504] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:42.105897 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:42.107583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.164344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e8e57-bc4a-47a3-b385-df4e86be2882 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:42.168598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a594d-6b4e-4ae4-a059-9fed1c547e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:42.169243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a594d-6b4e-4ae4-a059-9fed1c547e4f service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.169531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a594d-6b4e-4ae4-a059-9fed1c547e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.169814 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a594d-6b4e-4ae4-a059-9fed1c547e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.178901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a594d-6b4e-4ae4-a059-9fed1c547e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:42.180391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85a594d-6b4e-4ae4-a059-9fed1c547e4f service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:42.190626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1264/2505] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:42.333124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f80ea-f6c2-4af2-8222-bd0eec1e3418 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:42.395228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f80ea-f6c2-4af2-8222-bd0eec1e3418 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:42.400719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:42.401715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.402002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.402279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] PATH_INFO: `/v3/users/02c649fa52ad454f920689a2aed4a020` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.403478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c649fa52ad454f920689a2aed4a020)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:42.405518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:42.559874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c649fa52ad454f920689a2aed4a020'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.632050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c649fa52ad454f920689a2aed4a020'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.632696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c649fa52ad454f920689a2aed4a020'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.640619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c649fa52ad454f920689a2aed4a020'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.641248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c649fa52ad454f920689a2aed4a020'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:42.659429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1242/2506] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:42 2022] DELETE /identity/v3/users/02c649fa52ad454f920689a2aed4a020 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:42.663978 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb725a4-d854-41a4-8650-7a65df22cf4d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.664305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb725a4-d854-41a4-8650-7a65df22cf4d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.664586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb725a4-d854-41a4-8650-7a65df22cf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.681997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4cb725a4-d854-41a4-8650-7a65df22cf4d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:42.683193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4cb725a4-d854-41a4-8650-7a65df22cf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.740850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1265/2507] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:42.750501 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:42.751941 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:42.787066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:42.809006 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b93c6-523a-4db4-8d5d-16d9cb836317 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:42.812702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57aeecc-a308-4fe0-b4d0-7d323363c2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:42.813283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57aeecc-a308-4fe0-b4d0-7d323363c2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:42.813563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57aeecc-a308-4fe0-b4d0-7d323363c2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:42.813845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57aeecc-a308-4fe0-b4d0-7d323363c2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:42.823775 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57aeecc-a308-4fe0-b4d0-7d323363c2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:42.825235 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57aeecc-a308-4fe0-b4d0-7d323363c2a2 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:42.834414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1243/2508] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:43.269096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb725a4-d854-41a4-8650-7a65df22cf4d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:43.270541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4cb725a4-d854-41a4-8650-7a65df22cf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:43.291767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb725a4-d854-41a4-8650-7a65df22cf4d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:43.297696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:43.298590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:43.298873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:43.299161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] PATH_INFO: `/v3/projects/36cd278953ab46c08b1eb2cea55be8f4` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:43.300510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36cd278953ab46c08b1eb2cea55be8f4)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:43.302479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:43.326123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:43.423021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36cd278953ab46c08b1eb2cea55be8f4', 'request_id': 'req-121d8d9a-ce13-472e-bee9-46eba4389dc2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.463622 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36cd278953ab46c08b1eb2cea55be8f4', 'request_id': 'req-121d8d9a-ce13-472e-bee9-46eba4389dc2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.469602 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36cd278953ab46c08b1eb2cea55be8f4', 'request_id': 'req-121d8d9a-ce13-472e-bee9-46eba4389dc2'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.482138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] The token cache is being invalidate because project 36cd278953ab46c08b1eb2cea55be8f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:43.482624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.483448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.485203 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1266/2509] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:43 2022] DELETE /identity/v3/projects/36cd278953ab46c08b1eb2cea55be8f4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:43.489556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57aeecc-a308-4fe0-b4d0-7d323363c2a2 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:43.507909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57aeecc-a308-4fe0-b4d0-7d323363c2a2 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:43.590130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:43.590803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:43.591149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:43.591433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] PATH_INFO: `/v3/users/b8e5457d3d2a423785c69b2da1f61966` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:43.592690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8e5457d3d2a423785c69b2da1f61966)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:43.594625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:43.772663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8e5457d3d2a423785c69b2da1f61966'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.856155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8e5457d3d2a423785c69b2da1f61966'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.856730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8e5457d3d2a423785c69b2da1f61966'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.864001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8e5457d3d2a423785c69b2da1f61966'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.864692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-122b8c69-41c8-4dbc-a8ca-421aab2702a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8e5457d3d2a423785c69b2da1f61966'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:43.882470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1244/2510] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:43 2022] DELETE /identity/v3/users/b8e5457d3d2a423785c69b2da1f61966 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:44.350026 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:44.351757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:44.365079 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d8d9a-ce13-472e-bee9-46eba4389dc2 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:44.423570 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:44.424475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.424762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.425147 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] PATH_INFO: `/v3/projects/00b52f9c7b09424ab4fba0ee1079642e` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.426349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b52f9c7b09424ab4fba0ee1079642e)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:44.428361 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:44.531584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b52f9c7b09424ab4fba0ee1079642e', 'request_id': 'req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.594382 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b52f9c7b09424ab4fba0ee1079642e', 'request_id': 'req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.601087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b52f9c7b09424ab4fba0ee1079642e', 'request_id': 'req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.612837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] The token cache is being invalidate because project 00b52f9c7b09424ab4fba0ee1079642e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:44.613240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.613910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ae52b3ec-44fa-40ab-8a8a-d989ce60dfe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:44.615422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1267/2511] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:44 2022] DELETE /identity/v3/projects/00b52f9c7b09424ab4fba0ee1079642e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:44.622996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ef867-ff5d-4569-af09-972752b67d2b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.623201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ef867-ff5d-4569-af09-972752b67d2b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.623447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ef867-ff5d-4569-af09-972752b67d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.642979 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a80ef867-ff5d-4569-af09-972752b67d2b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:44.644317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a80ef867-ff5d-4569-af09-972752b67d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:44.710876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:44.724754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1245/2512] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:44.733944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79a85-e9ca-4e2d-b7cf-09a6d1e3f9b2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.734269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79a85-e9ca-4e2d-b7cf-09a6d1e3f9b2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.734602 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e79a85-e9ca-4e2d-b7cf-09a6d1e3f9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.753619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-75e79a85-e9ca-4e2d-b7cf-09a6d1e3f9b2 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:44.754837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-75e79a85-e9ca-4e2d-b7cf-09a6d1e3f9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:44.763729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1268/2513] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:44.768084 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ef867-ff5d-4569-af09-972752b67d2b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:44.778241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ef867-ff5d-4569-af09-972752b67d2b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:44.782935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:44.783589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.783873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.784186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.784911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:44.785225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:44.786898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:44.804046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1246/2514] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:06:44.804165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e79a85-e9ca-4e2d-b7cf-09a6d1e3f9b2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:44.814234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e79a85-e9ca-4e2d-b7cf-09a6d1e3f9b2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:44.818981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048caf9d-dfb8-4a86-9802-df20a87ad1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:44.819647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048caf9d-dfb8-4a86-9802-df20a87ad1be admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:44.819932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048caf9d-dfb8-4a86-9802-df20a87ad1be admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:44.820245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048caf9d-dfb8-4a86-9802-df20a87ad1be admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:44.821080 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048caf9d-dfb8-4a86-9802-df20a87ad1be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:44.822858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048caf9d-dfb8-4a86-9802-df20a87ad1be admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:44.943112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1269/2515] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:44 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:44.946984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:44.957954 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb218c0-7ed9-4f32-9cd2-f8da0afc55bb admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.016494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473550d-6d5c-463b-9d99-f4f2e6b5acb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.017185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473550d-6d5c-463b-9d99-f4f2e6b5acb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.017475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473550d-6d5c-463b-9d99-f4f2e6b5acb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.017757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473550d-6d5c-463b-9d99-f4f2e6b5acb0 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.018460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473550d-6d5c-463b-9d99-f4f2e6b5acb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.020418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473550d-6d5c-463b-9d99-f4f2e6b5acb0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.112626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1247/2516] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:44 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:45.116905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048caf9d-dfb8-4a86-9802-df20a87ad1be admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.127838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048caf9d-dfb8-4a86-9802-df20a87ad1be admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.132555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36075b-53ef-48d8-867f-153a12e087da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.133217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36075b-53ef-48d8-867f-153a12e087da admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.133497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36075b-53ef-48d8-867f-153a12e087da admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.133778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36075b-53ef-48d8-867f-153a12e087da admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.134531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36075b-53ef-48d8-867f-153a12e087da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.136414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36075b-53ef-48d8-867f-153a12e087da admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.138448 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:45.147944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1270/2517] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:45.152810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473550d-6d5c-463b-9d99-f4f2e6b5acb0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.162920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473550d-6d5c-463b-9d99-f4f2e6b5acb0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.168368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad2eec-3ab0-43e9-b92e-c1ebec05d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.169027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad2eec-3ab0-43e9-b92e-c1ebec05d931 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.169313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad2eec-3ab0-43e9-b92e-c1ebec05d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.169591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad2eec-3ab0-43e9-b92e-c1ebec05d931 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.170371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad2eec-3ab0-43e9-b92e-c1ebec05d931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.171962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad2eec-3ab0-43e9-b92e-c1ebec05d931 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.182117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1248/2518] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:45.186279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36075b-53ef-48d8-867f-153a12e087da admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.196859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36075b-53ef-48d8-867f-153a12e087da admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.201882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d05eb8-0bab-49dd-b9fc-f58e4d33e629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.202816 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d05eb8-0bab-49dd-b9fc-f58e4d33e629 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.203103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d05eb8-0bab-49dd-b9fc-f58e4d33e629 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.203378 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d05eb8-0bab-49dd-b9fc-f58e4d33e629 admin admin] PATH_INFO: `/v3/projects/e4565edad45b452c8208216bb607f866/users/049bd0789d1e42ea92236b57a1130cb1/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.218599 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d05eb8-0bab-49dd-b9fc-f58e4d33e629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4565edad45b452c8208216bb607f866, user_id=049bd0789d1e42ea92236b57a1130cb1, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.221302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d05eb8-0bab-49dd-b9fc-f58e4d33e629 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.310202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1271/2519] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:45 2022] PUT /identity/v3/projects/e4565edad45b452c8208216bb607f866/users/049bd0789d1e42ea92236b57a1130cb1/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:45.320594 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271a077-bfa8-406f-a46c-12f5a398f795 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.320887 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271a077-bfa8-406f-a46c-12f5a398f795 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.321276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271a077-bfa8-406f-a46c-12f5a398f795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.348764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b271a077-bfa8-406f-a46c-12f5a398f795 None None] MFA Rules not processed for user `049bd0789d1e42ea92236b57a1130cb1`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:45.349963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b271a077-bfa8-406f-a46c-12f5a398f795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:45.414472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1249/2520] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:45.419989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d05eb8-0bab-49dd-b9fc-f58e4d33e629 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.480823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d05eb8-0bab-49dd-b9fc-f58e4d33e629 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.485416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3f995-e6e6-4b10-b93c-edee094fb156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.486074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3f995-e6e6-4b10-b93c-edee094fb156 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.486355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3f995-e6e6-4b10-b93c-edee094fb156 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.486626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e3f995-e6e6-4b10-b93c-edee094fb156 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.487304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3f995-e6e6-4b10-b93c-edee094fb156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.489229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e3f995-e6e6-4b10-b93c-edee094fb156 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.594343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1272/2521] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:45 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:45.598544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271a077-bfa8-406f-a46c-12f5a398f795 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.609689 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271a077-bfa8-406f-a46c-12f5a398f795 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.654098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:45.665644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584aa3d-f31c-46d6-9e4a-b26decbe9a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.666317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584aa3d-f31c-46d6-9e4a-b26decbe9a07 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.666601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584aa3d-f31c-46d6-9e4a-b26decbe9a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.666879 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584aa3d-f31c-46d6-9e4a-b26decbe9a07 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.667592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584aa3d-f31c-46d6-9e4a-b26decbe9a07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.669533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584aa3d-f31c-46d6-9e4a-b26decbe9a07 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.787387 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1250/2522] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:45 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:45.791353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3f995-e6e6-4b10-b93c-edee094fb156 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.802064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e3f995-e6e6-4b10-b93c-edee094fb156 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.806617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df23e47-63a2-4f4c-a90d-ef36b914ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.807272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df23e47-63a2-4f4c-a90d-ef36b914ab01 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.807590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df23e47-63a2-4f4c-a90d-ef36b914ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.807969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df23e47-63a2-4f4c-a90d-ef36b914ab01 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.808844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df23e47-63a2-4f4c-a90d-ef36b914ab01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.810423 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df23e47-63a2-4f4c-a90d-ef36b914ab01 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.820787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1273/2523] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:45.825974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584aa3d-f31c-46d6-9e4a-b26decbe9a07 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.835765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584aa3d-f31c-46d6-9e4a-b26decbe9a07 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:45.841412 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65655ef6-14c2-4bd8-862f-117cc2722e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:45.842342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65655ef6-14c2-4bd8-862f-117cc2722e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:45.842654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65655ef6-14c2-4bd8-862f-117cc2722e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:45.842932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65655ef6-14c2-4bd8-862f-117cc2722e32 admin admin] PATH_INFO: `/v3/projects/5dbe26a99c9349c5ac7064a4f61e83cf/users/951725f5ac804f8e8bf4b0534476f9c8/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:45.860328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65655ef6-14c2-4bd8-862f-117cc2722e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbe26a99c9349c5ac7064a4f61e83cf, user_id=951725f5ac804f8e8bf4b0534476f9c8, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:45.865277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65655ef6-14c2-4bd8-862f-117cc2722e32 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:45.970179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1251/2524] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:45 2022] PUT /identity/v3/projects/5dbe26a99c9349c5ac7064a4f61e83cf/users/951725f5ac804f8e8bf4b0534476f9c8/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:45.974972 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df23e47-63a2-4f4c-a90d-ef36b914ab01 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:45.987338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df23e47-63a2-4f4c-a90d-ef36b914ab01 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.049719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383fd23b-7d81-42a0-aa79-96445d1d55f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.054841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fd23b-7d81-42a0-aa79-96445d1d55f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.054841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fd23b-7d81-42a0-aa79-96445d1d55f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.054841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fd23b-7d81-42a0-aa79-96445d1d55f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.054841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383fd23b-7d81-42a0-aa79-96445d1d55f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.054841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383fd23b-7d81-42a0-aa79-96445d1d55f5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.064956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1274/2525] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:46.069137 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65655ef6-14c2-4bd8-862f-117cc2722e32 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:46.083198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65655ef6-14c2-4bd8-862f-117cc2722e32 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.088275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.089727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.089727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.090076 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] PATH_INFO: `/v3/projects/5dbe26a99c9349c5ac7064a4f61e83cf/users/951725f5ac804f8e8bf4b0534476f9c8/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.093530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dbe26a99c9349c5ac7064a4f61e83cf, user_id=951725f5ac804f8e8bf4b0534476f9c8, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.096444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.108348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1252/2526] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:46 2022] PUT /identity/v3/projects/5dbe26a99c9349c5ac7064a4f61e83cf/users/951725f5ac804f8e8bf4b0534476f9c8/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:46.120638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003ebc-05b2-4ecc-8c62-50b58d78b9d2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.120941 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003ebc-05b2-4ecc-8c62-50b58d78b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.121227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003ebc-05b2-4ecc-8c62-50b58d78b9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.148657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-59003ebc-05b2-4ecc-8c62-50b58d78b9d2 None None] MFA Rules not processed for user `951725f5ac804f8e8bf4b0534476f9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:46.150162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-59003ebc-05b2-4ecc-8c62-50b58d78b9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:46.182188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:46.224213 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1275/2527] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:46.236208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:46.238110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:46.297958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff504ee8-f4a2-47ef-8996-58a7c6b26419 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:46.301685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ff4c9-8c38-4139-bff1-366507e539b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:46.306296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ff4c9-8c38-4139-bff1-366507e539b9 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:46.306296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ff4c9-8c38-4139-bff1-366507e539b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:46.306296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ff4c9-8c38-4139-bff1-366507e539b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:46.313111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ff4c9-8c38-4139-bff1-366507e539b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:46.314623 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377ff4c9-8c38-4139-bff1-366507e539b9 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:46.324930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1253/2528] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:47.257998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003ebc-05b2-4ecc-8c62-50b58d78b9d2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:47.328358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003ebc-05b2-4ecc-8c62-50b58d78b9d2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:47.333105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:47.333991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:47.334280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:47.334548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] PATH_INFO: `/v3/users/049bd0789d1e42ea92236b57a1130cb1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:47.335762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] RBAC: Authorizing `identity:delete_user(user_id=049bd0789d1e42ea92236b57a1130cb1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:47.337869 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:47.488886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '049bd0789d1e42ea92236b57a1130cb1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.549776 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '049bd0789d1e42ea92236b57a1130cb1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.550407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '049bd0789d1e42ea92236b57a1130cb1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.559102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '049bd0789d1e42ea92236b57a1130cb1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.559102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '049bd0789d1e42ea92236b57a1130cb1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:47.579227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1276/2529] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:47 2022] DELETE /identity/v3/users/049bd0789d1e42ea92236b57a1130cb1 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:47.584144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bac1b4-9677-47d3-9733-5eff0004a90c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:47.584640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bac1b4-9677-47d3-9733-5eff0004a90c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:47.584764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bac1b4-9677-47d3-9733-5eff0004a90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:47.602566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-17bac1b4-9677-47d3-9733-5eff0004a90c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:47.603706 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-17bac1b4-9677-47d3-9733-5eff0004a90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:47.664231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1254/2530] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:47.675339 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:47.676775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:47.734305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:47.738181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67857a31-45d7-44a9-b258-df9ae5faedae admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:47.741886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623dc57-afa7-4905-97d4-1ca5f313f1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:47.742537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623dc57-afa7-4905-97d4-1ca5f313f1cd service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:47.742944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623dc57-afa7-4905-97d4-1ca5f313f1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:47.743227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623dc57-afa7-4905-97d4-1ca5f313f1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:47.752963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623dc57-afa7-4905-97d4-1ca5f313f1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:47.754527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623dc57-afa7-4905-97d4-1ca5f313f1cd service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:47.764583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1277/2531] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:48.178260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bac1b4-9677-47d3-9733-5eff0004a90c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:48.178260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-17bac1b4-9677-47d3-9733-5eff0004a90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:48.191705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bac1b4-9677-47d3-9733-5eff0004a90c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:48.197721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:48.198603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:48.199037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:48.199209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] PATH_INFO: `/v3/projects/e4565edad45b452c8208216bb607f866` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:48.200420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4565edad45b452c8208216bb607f866)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:48.202427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:48.311821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:48.318585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4565edad45b452c8208216bb607f866', 'request_id': 'req-65d28afa-0385-45b3-b8f9-c3950fec3ff6'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.387208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4565edad45b452c8208216bb607f866', 'request_id': 'req-65d28afa-0385-45b3-b8f9-c3950fec3ff6'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.393976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4565edad45b452c8208216bb607f866', 'request_id': 'req-65d28afa-0385-45b3-b8f9-c3950fec3ff6'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.406005 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] The token cache is being invalidate because project e4565edad45b452c8208216bb607f866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:48.406325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.406975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.408595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1255/2532] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:48 2022] DELETE /identity/v3/projects/e4565edad45b452c8208216bb607f866 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:48.412701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623dc57-afa7-4905-97d4-1ca5f313f1cd service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:48.423561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623dc57-afa7-4905-97d4-1ca5f313f1cd service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:48.466246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:48.468932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:48.482572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:48.483455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:48.483757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:48.484039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] PATH_INFO: `/v3/users/951725f5ac804f8e8bf4b0534476f9c8` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:48.485279 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=951725f5ac804f8e8bf4b0534476f9c8)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:48.487216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:48.536595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d28afa-0385-45b3-b8f9-c3950fec3ff6 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:48.541255 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:48.542129 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:48.542405 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:48.542679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] PATH_INFO: `/v3/users/7c406ef1c63f49cabc03d5f2973d3a6e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:48.543884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c406ef1c63f49cabc03d5f2973d3a6e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:48.546188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:48.688442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '951725f5ac804f8e8bf4b0534476f9c8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.767281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c406ef1c63f49cabc03d5f2973d3a6e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.775801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '951725f5ac804f8e8bf4b0534476f9c8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.776469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '951725f5ac804f8e8bf4b0534476f9c8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.785376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '951725f5ac804f8e8bf4b0534476f9c8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.786019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-a3616c75-8879-4e5d-ba1c-4c99248141c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '951725f5ac804f8e8bf4b0534476f9c8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.803691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1278/2533] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:48 2022] DELETE /identity/v3/users/951725f5ac804f8e8bf4b0534476f9c8 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:48.849804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c406ef1c63f49cabc03d5f2973d3a6e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.851020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c406ef1c63f49cabc03d5f2973d3a6e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.861251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c406ef1c63f49cabc03d5f2973d3a6e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.862056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c406ef1c63f49cabc03d5f2973d3a6e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:48.886743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1256/2534] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:48 2022] DELETE /identity/v3/users/7c406ef1c63f49cabc03d5f2973d3a6e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:48.893681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227545a-0dd6-4866-9da1-729a28fcd48b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:48.893681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227545a-0dd6-4866-9da1-729a28fcd48b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:48.894062 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227545a-0dd6-4866-9da1-729a28fcd48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:48.919166 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d227545a-0dd6-4866-9da1-729a28fcd48b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:48.920611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d227545a-0dd6-4866-9da1-729a28fcd48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:48.981059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1279/2535] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:48.991032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:48.992580 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.052050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:49.058684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ebfcda-e899-4dbb-a604-3ef9db4ceb37 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:49.064514 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:49.065380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.065821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.066284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.080646 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:49.083622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:49.096467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1257/2536] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:49.275125 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227545a-0dd6-4866-9da1-729a28fcd48b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:49.277071 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d227545a-0dd6-4866-9da1-729a28fcd48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.291031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227545a-0dd6-4866-9da1-729a28fcd48b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:49.298321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:49.299477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.299766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.300099 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] PATH_INFO: `/v3/projects/5dbe26a99c9349c5ac7064a4f61e83cf` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.301815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dbe26a99c9349c5ac7064a4f61e83cf)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:49.304005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:49.313311 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:49.411805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:49.413421 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.413759 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dbe26a99c9349c5ac7064a4f61e83cf', 'request_id': 'req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.427025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f82592-fe99-4cd4-802e-35e3c1a9a60e service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:49.497525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:49.498990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.499279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.499560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] PATH_INFO: `/v3/users/c60d71f78d574be5a2ae6f925fd3bf65` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.500812 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60d71f78d574be5a2ae6f925fd3bf65)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:49.502831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:49.512406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dbe26a99c9349c5ac7064a4f61e83cf', 'request_id': 'req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.523187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dbe26a99c9349c5ac7064a4f61e83cf', 'request_id': 'req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.535376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] The token cache is being invalidate because project 5dbe26a99c9349c5ac7064a4f61e83cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:49.535676 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.536397 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-be201fb2-ab3c-4e3e-9c1b-22e41e2e8efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.537974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1280/2537] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:49 2022] DELETE /identity/v3/projects/5dbe26a99c9349c5ac7064a4f61e83cf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:49.542849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02621077-6a30-4960-82f9-9c2ac32284aa None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.543201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02621077-6a30-4960-82f9-9c2ac32284aa None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.544285 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02621077-6a30-4960-82f9-9c2ac32284aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.562774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-02621077-6a30-4960-82f9-9c2ac32284aa None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:49.564023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-02621077-6a30-4960-82f9-9c2ac32284aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.571987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1281/2538] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:49.580806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488185d2-60de-46a4-8875-6cb9ee808efd None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.581103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488185d2-60de-46a4-8875-6cb9ee808efd None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.581389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488185d2-60de-46a4-8875-6cb9ee808efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.600081 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-488185d2-60de-46a4-8875-6cb9ee808efd None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:49.601336 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-488185d2-60de-46a4-8875-6cb9ee808efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.610484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1282/2539] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:49.614558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488185d2-60de-46a4-8875-6cb9ee808efd None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:49.625630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488185d2-60de-46a4-8875-6cb9ee808efd None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:49.630838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:49.631518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.631805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.632087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.632869 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:49.633182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:49.634849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:49.647549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1283/2540] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:06:49.648432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:49.650417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.664533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2785d7-fdb1-42f6-82ed-5908f0426a34 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:49.669441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:49.670872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.671245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.671529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] PATH_INFO: `/v3/projects/a06728e0b0f6484cae09bcef4becc45d` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.672753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a06728e0b0f6484cae09bcef4becc45d)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:49.674726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:49.689670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60d71f78d574be5a2ae6f925fd3bf65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.754551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60d71f78d574be5a2ae6f925fd3bf65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.755149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60d71f78d574be5a2ae6f925fd3bf65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.762656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60d71f78d574be5a2ae6f925fd3bf65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.763294 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60d71f78d574be5a2ae6f925fd3bf65'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.784041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1258/2541] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:49 2022] DELETE /identity/v3/users/c60d71f78d574be5a2ae6f925fd3bf65 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:49.784752 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:49.785983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.804657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a06728e0b0f6484cae09bcef4becc45d', 'request_id': 'req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.849687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855d2d5-72cf-4a9c-b0db-cf02e1077576 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:49.861612 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:49.863328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a06728e0b0f6484cae09bcef4becc45d', 'request_id': 'req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.869580 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a06728e0b0f6484cae09bcef4becc45d', 'request_id': 'req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.881728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] The token cache is being invalidate because project a06728e0b0f6484cae09bcef4becc45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:49.882024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.882690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4b9605d7-2ebf-42e2-9abf-8bcf66ba3b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:49.884492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1284/2542] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:49 2022] DELETE /identity/v3/projects/a06728e0b0f6484cae09bcef4becc45d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:49.885731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d2ec7-7a28-4dba-a4b0-6cd9bb47d644 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.886024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d2ec7-7a28-4dba-a4b0-6cd9bb47d644 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.886302 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d2ec7-7a28-4dba-a4b0-6cd9bb47d644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.904052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-db8d2ec7-7a28-4dba-a4b0-6cd9bb47d644 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:49.906059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-db8d2ec7-7a28-4dba-a4b0-6cd9bb47d644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.907648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c60bf8-787b-492e-abb1-6fc332fac657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:49.908357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c60bf8-787b-492e-abb1-6fc332fac657 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.908649 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c60bf8-787b-492e-abb1-6fc332fac657 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.908935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c60bf8-787b-492e-abb1-6fc332fac657 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.909633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c60bf8-787b-492e-abb1-6fc332fac657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:49.911538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c60bf8-787b-492e-abb1-6fc332fac657 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:49.913739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1285/2543] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:49.914935 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128d566-67a2-48be-a274-834646df5a71 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:49.915220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128d566-67a2-48be-a274-834646df5a71 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:49.915812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128d566-67a2-48be-a274-834646df5a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:49.942685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2128d566-67a2-48be-a274-834646df5a71 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:49.943900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2128d566-67a2-48be-a274-834646df5a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.953110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1286/2544] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:49.954060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128d566-67a2-48be-a274-834646df5a71 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:49.955544 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2128d566-67a2-48be-a274-834646df5a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:49.996242 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1259/2545] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:49 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:49.996958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:50.001015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcfa5c-57d9-43ae-960e-17913958d848 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.001585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcfa5c-57d9-43ae-960e-17913958d848 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.002058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcfa5c-57d9-43ae-960e-17913958d848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.012763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128d566-67a2-48be-a274-834646df5a71 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.021499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4dbcfa5c-57d9-43ae-960e-17913958d848 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:50.023011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4dbcfa5c-57d9-43ae-960e-17913958d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.063144 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a613a-b008-4981-bd94-99f2558d83c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.063811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a613a-b008-4981-bd94-99f2558d83c0 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.064103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a613a-b008-4981-bd94-99f2558d83c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.064414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a613a-b008-4981-bd94-99f2558d83c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.085057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1260/2546] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:50.088967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcfa5c-57d9-43ae-960e-17913958d848 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.090733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4dbcfa5c-57d9-43ae-960e-17913958d848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.103197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcfa5c-57d9-43ae-960e-17913958d848 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.108909 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f06512-0564-4a79-9e95-98e51bf2c225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.109839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f06512-0564-4a79-9e95-98e51bf2c225 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.110138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f06512-0564-4a79-9e95-98e51bf2c225 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.111730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f06512-0564-4a79-9e95-98e51bf2c225 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.112448 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f06512-0564-4a79-9e95-98e51bf2c225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.114480 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f06512-0564-4a79-9e95-98e51bf2c225 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.128826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a613a-b008-4981-bd94-99f2558d83c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.130322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a613a-b008-4981-bd94-99f2558d83c0 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.140585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1287/2547] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:50.141374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a613a-b008-4981-bd94-99f2558d83c0 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.151584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a613a-b008-4981-bd94-99f2558d83c0 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.157631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.158559 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.158782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.159133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.159943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:50.160362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.162103 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.178551 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1288/2548] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:50.183606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.189855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908ef4e-3def-4b54-9c59-86769e0cb40a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.194751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a8453-073a-4cd1-a729-c2e87e6d7f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.195476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a8453-073a-4cd1-a729-c2e87e6d7f06 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.195764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a8453-073a-4cd1-a729-c2e87e6d7f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.196044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a8453-073a-4cd1-a729-c2e87e6d7f06 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.196781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a8453-073a-4cd1-a729-c2e87e6d7f06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.198737 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a8453-073a-4cd1-a729-c2e87e6d7f06 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.210307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1261/2549] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:50.214961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f06512-0564-4a79-9e95-98e51bf2c225 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.226604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f06512-0564-4a79-9e95-98e51bf2c225 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.231410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e2b68-4a58-4d8c-b57b-2338ef025f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.232085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2b68-4a58-4d8c-b57b-2338ef025f96 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.232430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2b68-4a58-4d8c-b57b-2338ef025f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.232716 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2b68-4a58-4d8c-b57b-2338ef025f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.233596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e2b68-4a58-4d8c-b57b-2338ef025f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.235198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e2b68-4a58-4d8c-b57b-2338ef025f96 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.245868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1262/2550] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:50.249878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e2b68-4a58-4d8c-b57b-2338ef025f96 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.259528 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e2b68-4a58-4d8c-b57b-2338ef025f96 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.265001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974b4a6-bbea-4bb9-8f2c-ac0f9a4bd072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.265602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974b4a6-bbea-4bb9-8f2c-ac0f9a4bd072 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.265882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974b4a6-bbea-4bb9-8f2c-ac0f9a4bd072 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.266182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974b4a6-bbea-4bb9-8f2c-ac0f9a4bd072 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.266890 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974b4a6-bbea-4bb9-8f2c-ac0f9a4bd072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.268632 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974b4a6-bbea-4bb9-8f2c-ac0f9a4bd072 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.279216 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1263/2551] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:50.288701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974b4a6-bbea-4bb9-8f2c-ac0f9a4bd072 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.305845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974b4a6-bbea-4bb9-8f2c-ac0f9a4bd072 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.306413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1289/2552] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:50.313045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a8453-073a-4cd1-a729-c2e87e6d7f06 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.390199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a8453-073a-4cd1-a729-c2e87e6d7f06 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.390600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f899a0e5-26dc-43d5-ab86-eafa59d874ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.391582 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a0e5-26dc-43d5-ab86-eafa59d874ab admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.391929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a0e5-26dc-43d5-ab86-eafa59d874ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.392306 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f899a0e5-26dc-43d5-ab86-eafa59d874ab admin admin] PATH_INFO: `/v3/projects/79f27d76abb74f8bbbae75a49e153526/users/28df35ff2d2d4d85b3990edee18232da/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.395705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:50.398146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321f97c-1112-4338-acfc-68357e84079e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.399393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321f97c-1112-4338-acfc-68357e84079e admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.399945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321f97c-1112-4338-acfc-68357e84079e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.400265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321f97c-1112-4338-acfc-68357e84079e admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.401310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321f97c-1112-4338-acfc-68357e84079e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.403027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321f97c-1112-4338-acfc-68357e84079e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.409628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899a0e5-26dc-43d5-ab86-eafa59d874ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f27d76abb74f8bbbae75a49e153526, user_id=28df35ff2d2d4d85b3990edee18232da, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.412485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f899a0e5-26dc-43d5-ab86-eafa59d874ab admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.481399 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:50.487807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1290/2553] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:50.491692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321f97c-1112-4338-acfc-68357e84079e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.501796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321f97c-1112-4338-acfc-68357e84079e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.506407 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac28e7c-4ab4-47c0-9c67-ec1cad06f1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.507047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac28e7c-4ab4-47c0-9c67-ec1cad06f1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.507326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac28e7c-4ab4-47c0-9c67-ec1cad06f1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.507598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac28e7c-4ab4-47c0-9c67-ec1cad06f1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.508347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac28e7c-4ab4-47c0-9c67-ec1cad06f1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.509914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac28e7c-4ab4-47c0-9c67-ec1cad06f1b0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.524395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1291/2554] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:50.524588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1264/2555] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:50 2022] PUT /identity/v3/projects/79f27d76abb74f8bbbae75a49e153526/users/28df35ff2d2d4d85b3990edee18232da/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:50.527222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac28e7c-4ab4-47c0-9c67-ec1cad06f1b0 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.533265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178fe9a-d91c-4d29-a72e-c23e096f3caa None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.533565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178fe9a-d91c-4d29-a72e-c23e096f3caa None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.533853 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178fe9a-d91c-4d29-a72e-c23e096f3caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.538090 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac28e7c-4ab4-47c0-9c67-ec1cad06f1b0 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.563156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7178fe9a-d91c-4d29-a72e-c23e096f3caa None None] MFA Rules not processed for user `28df35ff2d2d4d85b3990edee18232da`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:50.564652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7178fe9a-d91c-4d29-a72e-c23e096f3caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.604097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.604805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.605096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.605375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.606113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.607793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.618047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1292/2556] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:50.619212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.620693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.636880 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1265/2557] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:50.637069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516d11f-b1d4-496c-a9ac-3ab42222cbb8 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.637172 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178fe9a-d91c-4d29-a72e-c23e096f3caa None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.639841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.640752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.641042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.641325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] PATH_INFO: `/v3/projects/ab50d48ef49041c08031ba837a13f624` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.642537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab50d48ef49041c08031ba837a13f624)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.644496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.648253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178fe9a-d91c-4d29-a72e-c23e096f3caa None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.653159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1651f6-e1fa-403b-85f2-84ea1ad73999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.654049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1651f6-e1fa-403b-85f2-84ea1ad73999 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.654353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1651f6-e1fa-403b-85f2-84ea1ad73999 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.655250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1651f6-e1fa-403b-85f2-84ea1ad73999 admin admin] PATH_INFO: `/v3/projects/d275686dbcfc44769065d5d9ba1a6f0c/users/86335999e9744e4c815a628607618b4f/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.670134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1651f6-e1fa-403b-85f2-84ea1ad73999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d275686dbcfc44769065d5d9ba1a6f0c, user_id=86335999e9744e4c815a628607618b4f, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.673247 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1651f6-e1fa-403b-85f2-84ea1ad73999 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.766332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1266/2558] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:50 2022] PUT /identity/v3/projects/d275686dbcfc44769065d5d9ba1a6f0c/users/86335999e9744e4c815a628607618b4f/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:50.770557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1651f6-e1fa-403b-85f2-84ea1ad73999 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.774839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab50d48ef49041c08031ba837a13f624', 'request_id': 'req-5edaa8c6-f9a8-4887-bf38-2406d0011c40'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.832104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1651f6-e1fa-403b-85f2-84ea1ad73999 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.849418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab50d48ef49041c08031ba837a13f624', 'request_id': 'req-5edaa8c6-f9a8-4887-bf38-2406d0011c40'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.855790 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab50d48ef49041c08031ba837a13f624', 'request_id': 'req-5edaa8c6-f9a8-4887-bf38-2406d0011c40'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.868017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] The token cache is being invalidate because project ab50d48ef49041c08031ba837a13f624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:50.868344 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.868989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-5edaa8c6-f9a8-4887-bf38-2406d0011c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:50.870638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1293/2559] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:50 2022] DELETE /identity/v3/projects/ab50d48ef49041c08031ba837a13f624 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:50.872181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef235a-3fe5-4a8f-b734-c9890e1dae69 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.872707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef235a-3fe5-4a8f-b734-c9890e1dae69 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.873800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef235a-3fe5-4a8f-b734-c9890e1dae69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.883015 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:50.888755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.889538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.889834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.890116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.890606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f0ef235a-3fe5-4a8f-b734-c9890e1dae69 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:50.890817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.891810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f0ef235a-3fe5-4a8f-b734-c9890e1dae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.892760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.900199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1294/2560] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:50.901280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef235a-3fe5-4a8f-b734-c9890e1dae69 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.902747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f0ef235a-3fe5-4a8f-b734-c9890e1dae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:50.915346 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef235a-3fe5-4a8f-b734-c9890e1dae69 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:50.921504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:50.922476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:50.922763 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:50.923039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] PATH_INFO: `/v3/users/f962481419d34327b21cd736b065f313` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:50.924274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f962481419d34327b21cd736b065f313)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:50.926207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:50.983224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1267/2561] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:50 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:50.984182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:50.985834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.042645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b690e-01e7-4348-92b7-0328c132fa5f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.046831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.047476 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.047761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.048037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.089202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f962481419d34327b21cd736b065f313'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:51.109503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.110980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.120862 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1268/2562] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:51.121703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.123064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.135447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9fa8e-f153-4831-8173-f4151b64801b service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.141309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bab1-429e-4cd5-91db-ceaf5cf5f223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.142007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bab1-429e-4cd5-91db-ceaf5cf5f223 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.142502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bab1-429e-4cd5-91db-ceaf5cf5f223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.142622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9bab1-429e-4cd5-91db-ceaf5cf5f223 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.143250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9bab1-429e-4cd5-91db-ceaf5cf5f223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.145218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9bab1-429e-4cd5-91db-ceaf5cf5f223 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.158335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f962481419d34327b21cd736b065f313'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:51.162526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f962481419d34327b21cd736b065f313'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:51.162526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:51.166753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f962481419d34327b21cd736b065f313'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:51.167388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f962481419d34327b21cd736b065f313'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:51.184746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1295/2563] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:50 2022] DELETE /identity/v3/users/f962481419d34327b21cd736b065f313 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:51.234885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1269/2564] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:51 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:51.238900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.240434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.306108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5a3c6-a149-451b-90d4-e4ca6da76687 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.311028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e91b96-431c-4611-b317-4a2d2ec391b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.311756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e91b96-431c-4611-b317-4a2d2ec391b5 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.312438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e91b96-431c-4611-b317-4a2d2ec391b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.313333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e91b96-431c-4611-b317-4a2d2ec391b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.314666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e91b96-431c-4611-b317-4a2d2ec391b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.316526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e91b96-431c-4611-b317-4a2d2ec391b5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.327353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1296/2565] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:51.331621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bab1-429e-4cd5-91db-ceaf5cf5f223 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.342493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9bab1-429e-4cd5-91db-ceaf5cf5f223 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.347859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6ca39-7fe1-4229-b0aa-4fbc79c3eaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.348787 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6ca39-7fe1-4229-b0aa-4fbc79c3eaa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.349079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6ca39-7fe1-4229-b0aa-4fbc79c3eaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.349362 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6ca39-7fe1-4229-b0aa-4fbc79c3eaa5 admin admin] PATH_INFO: `/v3/projects/4a366549bfd24e11b503c01eba1225ff/users/98b9e6be6148465d823a0f40a94c3607/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.365165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6ca39-7fe1-4229-b0aa-4fbc79c3eaa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a366549bfd24e11b503c01eba1225ff, user_id=98b9e6be6148465d823a0f40a94c3607, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.367842 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6ca39-7fe1-4229-b0aa-4fbc79c3eaa5 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.473995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1270/2566] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:51 2022] PUT /identity/v3/projects/4a366549bfd24e11b503c01eba1225ff/users/98b9e6be6148465d823a0f40a94c3607/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:51.477956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e91b96-431c-4611-b317-4a2d2ec391b5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.487951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e91b96-431c-4611-b317-4a2d2ec391b5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.554028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a12d26-43c5-40c5-9db3-f468e6ef5141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.554767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a12d26-43c5-40c5-9db3-f468e6ef5141 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.555060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a12d26-43c5-40c5-9db3-f468e6ef5141 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.555340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a12d26-43c5-40c5-9db3-f468e6ef5141 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.556322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a12d26-43c5-40c5-9db3-f468e6ef5141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.558040 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a12d26-43c5-40c5-9db3-f468e6ef5141 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.570135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1297/2567] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:51.574300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6ca39-7fe1-4229-b0aa-4fbc79c3eaa5 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.590517 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6ca39-7fe1-4229-b0aa-4fbc79c3eaa5 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.601196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.602261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.602529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.602811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] PATH_INFO: `/v3/projects/4a366549bfd24e11b503c01eba1225ff/users/98b9e6be6148465d823a0f40a94c3607/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.604992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a366549bfd24e11b503c01eba1225ff, user_id=98b9e6be6148465d823a0f40a94c3607, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.607978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.620883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1271/2568] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:51 2022] PUT /identity/v3/projects/4a366549bfd24e11b503c01eba1225ff/users/98b9e6be6148465d823a0f40a94c3607/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:06:51.632431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f646a7c-3d9a-442a-bca7-9960aa2fb898 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.632794 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f646a7c-3d9a-442a-bca7-9960aa2fb898 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.639462 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f646a7c-3d9a-442a-bca7-9960aa2fb898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.662101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9f646a7c-3d9a-442a-bca7-9960aa2fb898 None None] MFA Rules not processed for user `98b9e6be6148465d823a0f40a94c3607`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:51.662899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9f646a7c-3d9a-442a-bca7-9960aa2fb898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.715714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:51.727824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.731107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.747348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1298/2569] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:51.763091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f646a7c-3d9a-442a-bca7-9960aa2fb898 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:51.763091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9f646a7c-3d9a-442a-bca7-9960aa2fb898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.809820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:51.841211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba374ecd-8085-41e8-adf7-e72c38b3370c admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.842648 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f646a7c-3d9a-442a-bca7-9960aa2fb898 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:51.846249 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.846420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676b45a-7bdc-427f-9ea5-894368ce27fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:51.847037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676b45a-7bdc-427f-9ea5-894368ce27fd service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.847149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:51.847322 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676b45a-7bdc-427f-9ea5-894368ce27fd service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.847432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:51.847604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676b45a-7bdc-427f-9ea5-894368ce27fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.847864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] PATH_INFO: `/v3/projects/51cad71a48844196955f5e61efa9a2bf` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:51.849132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] RBAC: Authorizing `identity:delete_project(project_id=51cad71a48844196955f5e61efa9a2bf)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.849478 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e676b45a-7bdc-427f-9ea5-894368ce27fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:51.851045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.911538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676b45a-7bdc-427f-9ea5-894368ce27fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:51.913156 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676b45a-7bdc-427f-9ea5-894368ce27fd service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:51.922447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1299/2570] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:51.959877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51cad71a48844196955f5e61efa9a2bf', 'request_id': 'req-cb190a28-8411-4305-91ba-e7c12377c58f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:52.035166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51cad71a48844196955f5e61efa9a2bf', 'request_id': 'req-cb190a28-8411-4305-91ba-e7c12377c58f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:52.043750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51cad71a48844196955f5e61efa9a2bf', 'request_id': 'req-cb190a28-8411-4305-91ba-e7c12377c58f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:52.056461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] The token cache is being invalidate because project 51cad71a48844196955f5e61efa9a2bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:06:52.056755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:52.057440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-cb190a28-8411-4305-91ba-e7c12377c58f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:52.059265 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1272/2571] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:06:51 2022] DELETE /identity/v3/projects/51cad71a48844196955f5e61efa9a2bf => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:52.064723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab707d2-b62a-4294-a842-224a4f3db701 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.064948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab707d2-b62a-4294-a842-224a4f3db701 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.065231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab707d2-b62a-4294-a842-224a4f3db701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.084065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9ab707d2-b62a-4294-a842-224a4f3db701 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:52.085305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9ab707d2-b62a-4294-a842-224a4f3db701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:52.145502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1300/2572] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:52.153761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbda56-1bfe-499c-a32c-1444611cfb11 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.154153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbda56-1bfe-499c-a32c-1444611cfb11 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.154565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbda56-1bfe-499c-a32c-1444611cfb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.172968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-18dbda56-1bfe-499c-a32c-1444611cfb11 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:52.174177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-18dbda56-1bfe-499c-a32c-1444611cfb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:52.182449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1273/2573] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:06:52.187191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab707d2-b62a-4294-a842-224a4f3db701 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:52.197023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab707d2-b62a-4294-a842-224a4f3db701 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:52.202385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:52.203277 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.204191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.204191 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.204907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:06:52.204907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:52.206520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:52.219530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1301/2574] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:06:52.226267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbda56-1bfe-499c-a32c-1444611cfb11 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:52.237661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbda56-1bfe-499c-a32c-1444611cfb11 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:52.242538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff647e6-f4f0-4d87-8e8a-b3cc95d32fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:52.243206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff647e6-f4f0-4d87-8e8a-b3cc95d32fa1 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.243495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff647e6-f4f0-4d87-8e8a-b3cc95d32fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.245426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff647e6-f4f0-4d87-8e8a-b3cc95d32fa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.245426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff647e6-f4f0-4d87-8e8a-b3cc95d32fa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:52.246620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff647e6-f4f0-4d87-8e8a-b3cc95d32fa1 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:52.389086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1274/2575] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:52 2022] POST /identity/v3/projects => generated 393 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:52.393545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:52.403986 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b052977-75f3-4f52-95f8-8aaf7260e393 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:52.433129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:52.460348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf571d6-1c4e-449b-9cfc-f33695e7004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:52.461022 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf571d6-1c4e-449b-9cfc-f33695e7004c admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.461306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf571d6-1c4e-449b-9cfc-f33695e7004c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.461586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf571d6-1c4e-449b-9cfc-f33695e7004c admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.462278 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf571d6-1c4e-449b-9cfc-f33695e7004c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:52.464211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf571d6-1c4e-449b-9cfc-f33695e7004c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:52.564867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1302/2576] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:52 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:52.566379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff647e6-f4f0-4d87-8e8a-b3cc95d32fa1 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:52.576898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff647e6-f4f0-4d87-8e8a-b3cc95d32fa1 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:52.581721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cece7be-8243-4d22-bb59-9d3014d4dbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:52.593598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece7be-8243-4d22-bb59-9d3014d4dbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.593598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece7be-8243-4d22-bb59-9d3014d4dbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.593598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cece7be-8243-4d22-bb59-9d3014d4dbe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.593598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cece7be-8243-4d22-bb59-9d3014d4dbe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:52.594772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cece7be-8243-4d22-bb59-9d3014d4dbe8 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:52.617126 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1275/2577] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:52.625168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf571d6-1c4e-449b-9cfc-f33695e7004c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:52.645303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf571d6-1c4e-449b-9cfc-f33695e7004c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:52.652303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b98e0-709e-4712-b641-e291bace411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:52.653074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b98e0-709e-4712-b641-e291bace411e admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.653419 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b98e0-709e-4712-b641-e291bace411e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.653756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b98e0-709e-4712-b641-e291bace411e admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.656138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b98e0-709e-4712-b641-e291bace411e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:52.658019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b98e0-709e-4712-b641-e291bace411e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:52.669899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1303/2578] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:52.677949 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cece7be-8243-4d22-bb59-9d3014d4dbe8 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:52.688079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cece7be-8243-4d22-bb59-9d3014d4dbe8 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:52.693356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf3d79-1cf4-4902-832c-fb5fd5f94e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:52.694325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf3d79-1cf4-4902-832c-fb5fd5f94e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.694681 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf3d79-1cf4-4902-832c-fb5fd5f94e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.695028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf3d79-1cf4-4902-832c-fb5fd5f94e30 admin admin] PATH_INFO: `/v3/projects/fa4b13f31bd241f0b4c83bd7e1613c8b/users/89feae5654d345b68b1989c0c41dcda1/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.714795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf3d79-1cf4-4902-832c-fb5fd5f94e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4b13f31bd241f0b4c83bd7e1613c8b, user_id=89feae5654d345b68b1989c0c41dcda1, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:52.718331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf3d79-1cf4-4902-832c-fb5fd5f94e30 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:52.801824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1276/2579] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:52 2022] PUT /identity/v3/projects/fa4b13f31bd241f0b4c83bd7e1613c8b/users/89feae5654d345b68b1989c0c41dcda1/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:52.813586 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2225006-8c09-4201-8c3c-8b729291b98a None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.813878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2225006-8c09-4201-8c3c-8b729291b98a None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.814158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2225006-8c09-4201-8c3c-8b729291b98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.842356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-c2225006-8c09-4201-8c3c-8b729291b98a None None] MFA Rules not processed for user `89feae5654d345b68b1989c0c41dcda1`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:52.844174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c2225006-8c09-4201-8c3c-8b729291b98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:52.918787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1304/2580] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:52.924035 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf3d79-1cf4-4902-832c-fb5fd5f94e30 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:52.988789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf3d79-1cf4-4902-832c-fb5fd5f94e30 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:52.993751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b9667-6864-4017-ad11-f3cda9b771fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:52.994420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9667-6864-4017-ad11-f3cda9b771fe admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:52.994710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9667-6864-4017-ad11-f3cda9b771fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:52.994987 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9667-6864-4017-ad11-f3cda9b771fe admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:52.995748 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b9667-6864-4017-ad11-f3cda9b771fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:52.997806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b9667-6864-4017-ad11-f3cda9b771fe admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:53.006493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:53.095614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1277/2581] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:52 2022] POST /identity/v3/projects => generated 391 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:53.099848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2225006-8c09-4201-8c3c-8b729291b98a None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:53.113216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2225006-8c09-4201-8c3c-8b729291b98a None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:53.181496 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83387094-9a0f-4663-ad48-9e13851a32aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:53.182308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387094-9a0f-4663-ad48-9e13851a32aa admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:53.182668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387094-9a0f-4663-ad48-9e13851a32aa admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:53.183004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387094-9a0f-4663-ad48-9e13851a32aa admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:53.183878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83387094-9a0f-4663-ad48-9e13851a32aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:53.187061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83387094-9a0f-4663-ad48-9e13851a32aa admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:53.320138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1305/2582] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:53 2022] POST /identity/v3/users => generated 302 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:53.324063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b9667-6864-4017-ad11-f3cda9b771fe admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:53.346397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b9667-6864-4017-ad11-f3cda9b771fe admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:53.354401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431c6bd-e144-4295-bfd4-c332b93e6925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:53.355195 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431c6bd-e144-4295-bfd4-c332b93e6925 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:53.356043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431c6bd-e144-4295-bfd4-c332b93e6925 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:53.356426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431c6bd-e144-4295-bfd4-c332b93e6925 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:53.357664 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431c6bd-e144-4295-bfd4-c332b93e6925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:53.359380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5431c6bd-e144-4295-bfd4-c332b93e6925 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:53.374031 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1278/2583] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:06:53.378150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83387094-9a0f-4663-ad48-9e13851a32aa admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:53.389713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83387094-9a0f-4663-ad48-9e13851a32aa admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:53.395379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61f5b-0ac6-4b38-b968-0ea6b1b6e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:53.396421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61f5b-0ac6-4b38-b968-0ea6b1b6e0dc admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:53.396773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61f5b-0ac6-4b38-b968-0ea6b1b6e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:53.397121 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61f5b-0ac6-4b38-b968-0ea6b1b6e0dc admin admin] PATH_INFO: `/v3/projects/1606ecfe80ad4ae28a10ae872fb85661/users/631c17e3e3b24f6da61efe8fddfe5129/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:53.414914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61f5b-0ac6-4b38-b968-0ea6b1b6e0dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1606ecfe80ad4ae28a10ae872fb85661, user_id=631c17e3e3b24f6da61efe8fddfe5129, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:53.417983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61f5b-0ac6-4b38-b968-0ea6b1b6e0dc admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:53.538691 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1306/2584] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:53 2022] PUT /identity/v3/projects/1606ecfe80ad4ae28a10ae872fb85661/users/631c17e3e3b24f6da61efe8fddfe5129/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:53.542998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431c6bd-e144-4295-bfd4-c332b93e6925 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:53.556098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5431c6bd-e144-4295-bfd4-c332b93e6925 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:53.615441 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f824ded-2f38-4423-a89d-8a0f2da4b220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:53.616192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824ded-2f38-4423-a89d-8a0f2da4b220 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:53.616497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824ded-2f38-4423-a89d-8a0f2da4b220 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:53.616784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824ded-2f38-4423-a89d-8a0f2da4b220 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:53.617525 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824ded-2f38-4423-a89d-8a0f2da4b220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:53.619151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f824ded-2f38-4423-a89d-8a0f2da4b220 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:53.636433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1279/2585] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:53.640663 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61f5b-0ac6-4b38-b968-0ea6b1b6e0dc admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:53.649531 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:53.654761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61f5b-0ac6-4b38-b968-0ea6b1b6e0dc admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:53.659806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:53.660894 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:53.661019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:53.661269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] PATH_INFO: `/v3/projects/1606ecfe80ad4ae28a10ae872fb85661/users/631c17e3e3b24f6da61efe8fddfe5129/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:53.663266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1606ecfe80ad4ae28a10ae872fb85661, user_id=631c17e3e3b24f6da61efe8fddfe5129, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:53.666037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:53.678398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1307/2586] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:53 2022] PUT /identity/v3/projects/1606ecfe80ad4ae28a10ae872fb85661/users/631c17e3e3b24f6da61efe8fddfe5129/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:53.689014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66928e-a62f-49e6-ae4d-385f2d0751c2 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:53.689401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66928e-a62f-49e6-ae4d-385f2d0751c2 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:53.689577 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66928e-a62f-49e6-ae4d-385f2d0751c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:53.733434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4c66928e-a62f-49e6-ae4d-385f2d0751c2 None None] MFA Rules not processed for user `631c17e3e3b24f6da61efe8fddfe5129`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:53.734981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4c66928e-a62f-49e6-ae4d-385f2d0751c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:53.827704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1280/2587] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:53.842929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:53.844961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:53.906504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe8748-69b5-49fa-bf1e-a6d7a70428a8 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:53.910704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6835802-1d3e-4dad-9621-0503e7037348 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:53.913235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835802-1d3e-4dad-9621-0503e7037348 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:53.913235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835802-1d3e-4dad-9621-0503e7037348 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:53.913235 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835802-1d3e-4dad-9621-0503e7037348 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:53.921473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6835802-1d3e-4dad-9621-0503e7037348 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:53.922918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6835802-1d3e-4dad-9621-0503e7037348 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:53.932731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1308/2588] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:53.981395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66928e-a62f-49e6-ae4d-385f2d0751c2 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:53.992929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66928e-a62f-49e6-ae4d-385f2d0751c2 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:53.996934 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:53.997585 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:53.997871 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:53.998174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:54.073125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:54.074381 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:54.085602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1281/2589] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:06:54.298495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:57.366850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca42783-ea46-4a78-90ea-fad18d977e4b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:57.368268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca42783-ea46-4a78-90ea-fad18d977e4b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:57.368558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca42783-ea46-4a78-90ea-fad18d977e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:57.377118 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:57.378809 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:57.397533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780f880-74a0-45b4-8fd9-a3b8e84db3e9 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:57.409260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-aca42783-ea46-4a78-90ea-fad18d977e4b None None] MFA Rules not processed for user `86335999e9744e4c815a628607618b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:57.409260 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-aca42783-ea46-4a78-90ea-fad18d977e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:57.409919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:57.410604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:57.410896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:57.411181 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:57.415499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:57.536588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1309/2590] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:06:57.544198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca42783-ea46-4a78-90ea-fad18d977e4b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:57.546577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-aca42783-ea46-4a78-90ea-fad18d977e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:06:57.548020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:57.552154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:57.572178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1282/2591] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:06:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:06:57.639873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca42783-ea46-4a78-90ea-fad18d977e4b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:57.645347 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac60f-26e4-466f-bba0-d3eb079acfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:57.646150 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac60f-26e4-466f-bba0-d3eb079acfb8 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:57.646536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac60f-26e4-466f-bba0-d3eb079acfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:57.646939 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badac60f-26e4-466f-bba0-d3eb079acfb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:57.647741 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badac60f-26e4-466f-bba0-d3eb079acfb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:57.650174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badac60f-26e4-466f-bba0-d3eb079acfb8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:57.755053 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1310/2592] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:06:57 2022] POST /identity/v3/projects => generated 379 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:06:57.759191 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:57.778396 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4bdc76-08f5-4bf9-84cf-55d35e134f2c service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:57.848032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34954e4-a3f6-46d9-a69b-a01792a33ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:57.849029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34954e4-a3f6-46d9-a69b-a01792a33ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:57.849367 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34954e4-a3f6-46d9-a69b-a01792a33ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:57.849790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34954e4-a3f6-46d9-a69b-a01792a33ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:57.850543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34954e4-a3f6-46d9-a69b-a01792a33ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:57.852639 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34954e4-a3f6-46d9-a69b-a01792a33ca2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:58.054417 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1283/2593] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:06:57 2022] POST /identity/v3/users => generated 296 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:06:58.061102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac60f-26e4-466f-bba0-d3eb079acfb8 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:58.073607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badac60f-26e4-466f-bba0-d3eb079acfb8 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:58.078714 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49247611-e5e6-4360-b03b-27591adebf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:58.079429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247611-e5e6-4360-b03b-27591adebf88 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:58.079710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247611-e5e6-4360-b03b-27591adebf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:58.080009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247611-e5e6-4360-b03b-27591adebf88 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:58.080806 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247611-e5e6-4360-b03b-27591adebf88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:58.082413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247611-e5e6-4360-b03b-27591adebf88 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:58.085826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:06:58.100289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1311/2594] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:58.101813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34954e4-a3f6-46d9-a69b-a01792a33ca2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:58.114509 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34954e4-a3f6-46d9-a69b-a01792a33ca2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:58.119825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b433-0199-4f4d-8e90-f191f4308c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:58.120876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b433-0199-4f4d-8e90-f191f4308c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:58.121159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b433-0199-4f4d-8e90-f191f4308c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:58.121439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323b433-0199-4f4d-8e90-f191f4308c4d admin admin] PATH_INFO: `/v3/projects/b757855ab6ea423da9361a488f6cafd6/users/ee69b3f14a814f0aaa4de380bf2cc819/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:58.142997 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323b433-0199-4f4d-8e90-f191f4308c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b757855ab6ea423da9361a488f6cafd6, user_id=ee69b3f14a814f0aaa4de380bf2cc819, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:58.145831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323b433-0199-4f4d-8e90-f191f4308c4d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:58.260619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1284/2595] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:58 2022] PUT /identity/v3/projects/b757855ab6ea423da9361a488f6cafd6/users/ee69b3f14a814f0aaa4de380bf2cc819/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:58.265225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49247611-e5e6-4360-b03b-27591adebf88 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:58.284460 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49247611-e5e6-4360-b03b-27591adebf88 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:58.359132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1c3f4-e78e-4fe7-820a-4d346334c551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:58.360143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1c3f4-e78e-4fe7-820a-4d346334c551 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:58.360505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1c3f4-e78e-4fe7-820a-4d346334c551 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:58.360719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1c3f4-e78e-4fe7-820a-4d346334c551 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:58.362271 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1c3f4-e78e-4fe7-820a-4d346334c551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:58.363255 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1c3f4-e78e-4fe7-820a-4d346334c551 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:58.380922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1312/2596] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:06:58 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:06:58.384999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b433-0199-4f4d-8e90-f191f4308c4d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:58.402866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323b433-0199-4f4d-8e90-f191f4308c4d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:58.414470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b537ff-17b2-4cf9-97a6-de40b58adb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:58.416303 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b537ff-17b2-4cf9-97a6-de40b58adb12 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:58.416729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b537ff-17b2-4cf9-97a6-de40b58adb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:58.423299 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b537ff-17b2-4cf9-97a6-de40b58adb12 admin admin] PATH_INFO: `/v3/projects/b757855ab6ea423da9361a488f6cafd6/users/ee69b3f14a814f0aaa4de380bf2cc819/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:58.438370 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b537ff-17b2-4cf9-97a6-de40b58adb12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b757855ab6ea423da9361a488f6cafd6, user_id=ee69b3f14a814f0aaa4de380bf2cc819, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:58.446273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b537ff-17b2-4cf9-97a6-de40b58adb12 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:58.479445 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1285/2597] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:06:58 2022] PUT /identity/v3/projects/b757855ab6ea423da9361a488f6cafd6/users/ee69b3f14a814f0aaa4de380bf2cc819/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:59.503225 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1c3f4-e78e-4fe7-820a-4d346334c551 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:06:59.525435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1c3f4-e78e-4fe7-820a-4d346334c551 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:06:59.631562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:06:59.633268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:59.633792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:59.634170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] PATH_INFO: `/v3/users/89feae5654d345b68b1989c0c41dcda1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:59.637942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89feae5654d345b68b1989c0c41dcda1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:06:59.637942 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:06:59.832475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89feae5654d345b68b1989c0c41dcda1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:59.895479 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89feae5654d345b68b1989c0c41dcda1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:59.896108 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89feae5654d345b68b1989c0c41dcda1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:59.904207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89feae5654d345b68b1989c0c41dcda1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:59.905273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89feae5654d345b68b1989c0c41dcda1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:06:59.927786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1313/2598] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:06:59 2022] DELETE /identity/v3/users/89feae5654d345b68b1989c0c41dcda1 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:06:59.932363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fed844-b7ae-479a-902e-78963cf7a67c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:06:59.932363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fed844-b7ae-479a-902e-78963cf7a67c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:06:59.932940 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fed844-b7ae-479a-902e-78963cf7a67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:06:59.963745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-23fed844-b7ae-479a-902e-78963cf7a67c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:06:59.965350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-23fed844-b7ae-479a-902e-78963cf7a67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:00.011140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:00.038449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1286/2599] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:00.050209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:00.053131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:00.069762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:00.115966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c20ac7-fff6-4b9b-b62a-7390776cfe11 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:00.121613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abc05c-758e-412e-bc48-d7d09fca5b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:00.121613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abc05c-758e-412e-bc48-d7d09fca5b06 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:00.121613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abc05c-758e-412e-bc48-d7d09fca5b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:00.121613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abc05c-758e-412e-bc48-d7d09fca5b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:00.132327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abc05c-758e-412e-bc48-d7d09fca5b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:00.133782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abc05c-758e-412e-bc48-d7d09fca5b06 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:00.147028 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1314/2600] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:01.065438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fed844-b7ae-479a-902e-78963cf7a67c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:01.071333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-23fed844-b7ae-479a-902e-78963cf7a67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:01.103264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fed844-b7ae-479a-902e-78963cf7a67c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:01.111956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:01.114473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:01.115001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:01.115489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] PATH_INFO: `/v3/projects/fa4b13f31bd241f0b4c83bd7e1613c8b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:01.119120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4b13f31bd241f0b4c83bd7e1613c8b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:01.122071 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:01.352604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4b13f31bd241f0b4c83bd7e1613c8b', 'request_id': 'req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.452666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4b13f31bd241f0b4c83bd7e1613c8b', 'request_id': 'req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.458831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4b13f31bd241f0b4c83bd7e1613c8b', 'request_id': 'req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.471378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] The token cache is being invalidate because project fa4b13f31bd241f0b4c83bd7e1613c8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:01.471690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.472382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:01.473991 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1287/2601] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:01 2022] DELETE /identity/v3/projects/fa4b13f31bd241f0b4c83bd7e1613c8b => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:01.478068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abc05c-758e-412e-bc48-d7d09fca5b06 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:01.488078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abc05c-758e-412e-bc48-d7d09fca5b06 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:01.564001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:01.564960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:01.565248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:01.565521 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] PATH_INFO: `/v3/users/631c17e3e3b24f6da61efe8fddfe5129` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:01.566889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631c17e3e3b24f6da61efe8fddfe5129)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:01.570073 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:01.938161 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631c17e3e3b24f6da61efe8fddfe5129'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:02.021572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631c17e3e3b24f6da61efe8fddfe5129'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:02.022224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631c17e3e3b24f6da61efe8fddfe5129'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:02.032744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631c17e3e3b24f6da61efe8fddfe5129'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:02.033910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d7231866-6b91-4fb6-aa18-d760ec890869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631c17e3e3b24f6da61efe8fddfe5129'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:02.076655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1315/2602] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:01 2022] DELETE /identity/v3/users/631c17e3e3b24f6da61efe8fddfe5129 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:02.792712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:02.794344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:02.813132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e24fc0-8867-4611-9a8f-3ffaa5bac2e2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:02.938564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:02.939736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:02.940025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:02.940394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] PATH_INFO: `/v3/projects/1606ecfe80ad4ae28a10ae872fb85661` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:02.945277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1606ecfe80ad4ae28a10ae872fb85661)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:02.945277 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:03.096736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1606ecfe80ad4ae28a10ae872fb85661', 'request_id': 'req-91cfe646-c856-4665-b7e0-f9ba6a49e17f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:03.212079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1606ecfe80ad4ae28a10ae872fb85661', 'request_id': 'req-91cfe646-c856-4665-b7e0-f9ba6a49e17f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:03.214980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1606ecfe80ad4ae28a10ae872fb85661', 'request_id': 'req-91cfe646-c856-4665-b7e0-f9ba6a49e17f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:03.224198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:03.231846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] The token cache is being invalidate because project 1606ecfe80ad4ae28a10ae872fb85661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:03.233746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:03.234814 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-91cfe646-c856-4665-b7e0-f9ba6a49e17f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:03.236783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1288/2603] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:02 2022] DELETE /identity/v3/projects/1606ecfe80ad4ae28a10ae872fb85661 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:03.242785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac6e5f-2f4e-4f7c-8677-ec232373eb5d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.243327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac6e5f-2f4e-4f7c-8677-ec232373eb5d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.243442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac6e5f-2f4e-4f7c-8677-ec232373eb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.269418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-03ac6e5f-2f4e-4f7c-8677-ec232373eb5d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:03.269418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-03ac6e5f-2f4e-4f7c-8677-ec232373eb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:03.302049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:03.337107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1316/2604] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:03.345746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb0038-4959-48dc-8ad0-54ba0e540a53 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.346042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb0038-4959-48dc-8ad0-54ba0e540a53 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.346512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb0038-4959-48dc-8ad0-54ba0e540a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.368770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-88cb0038-4959-48dc-8ad0-54ba0e540a53 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:03.369999 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-88cb0038-4959-48dc-8ad0-54ba0e540a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:03.379598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1289/2605] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:03.384107 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac6e5f-2f4e-4f7c-8677-ec232373eb5d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:03.395717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac6e5f-2f4e-4f7c-8677-ec232373eb5d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:03.401206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:03.402023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.402404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.403068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.404014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:03.404489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:03.406495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:03.423701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1317/2606] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:07:03.427327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb0038-4959-48dc-8ad0-54ba0e540a53 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:03.438439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb0038-4959-48dc-8ad0-54ba0e540a53 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:03.443485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b79fda-40a2-4a36-b801-ba9376fd6236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:03.444384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b79fda-40a2-4a36-b801-ba9376fd6236 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.444847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b79fda-40a2-4a36-b801-ba9376fd6236 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.445278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b79fda-40a2-4a36-b801-ba9376fd6236 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.446155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b79fda-40a2-4a36-b801-ba9376fd6236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:03.448338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b79fda-40a2-4a36-b801-ba9376fd6236 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:03.574977 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1290/2607] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:03 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:03.579163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:03.595160 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edcef92-1eed-4235-ade9-a62c154f1cd5 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:03.687553 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c98c5e-484b-4b11-9aa9-ba5b6b9d85c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:03.688650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98c5e-484b-4b11-9aa9-ba5b6b9d85c1 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.689111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98c5e-484b-4b11-9aa9-ba5b6b9d85c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.689558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c98c5e-484b-4b11-9aa9-ba5b6b9d85c1 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.690474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c98c5e-484b-4b11-9aa9-ba5b6b9d85c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:03.692859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c98c5e-484b-4b11-9aa9-ba5b6b9d85c1 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:03.784661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1318/2608] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:03 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:03.788684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b79fda-40a2-4a36-b801-ba9376fd6236 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:03.801382 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b79fda-40a2-4a36-b801-ba9376fd6236 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:03.809106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185934b5-f815-4263-bc7b-388e6285ad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:03.809783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185934b5-f815-4263-bc7b-388e6285ad52 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.810064 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185934b5-f815-4263-bc7b-388e6285ad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.810364 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185934b5-f815-4263-bc7b-388e6285ad52 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.811708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185934b5-f815-4263-bc7b-388e6285ad52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:03.813489 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185934b5-f815-4263-bc7b-388e6285ad52 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:03.830256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1291/2609] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:03.834432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c98c5e-484b-4b11-9aa9-ba5b6b9d85c1 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:03.845969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c98c5e-484b-4b11-9aa9-ba5b6b9d85c1 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:03.853219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24120c33-c2ed-414d-864c-f51d050ad4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:03.853219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24120c33-c2ed-414d-864c-f51d050ad4fa admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.853219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24120c33-c2ed-414d-864c-f51d050ad4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.853219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24120c33-c2ed-414d-864c-f51d050ad4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.854398 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24120c33-c2ed-414d-864c-f51d050ad4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:03.858507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24120c33-c2ed-414d-864c-f51d050ad4fa admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:03.872706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1319/2610] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:03.876669 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185934b5-f815-4263-bc7b-388e6285ad52 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:03.895694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185934b5-f815-4263-bc7b-388e6285ad52 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:03.908167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:03.909198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:03.909399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:03.910403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] PATH_INFO: `/v3/projects/a098aacb8cc147ad8b22c3dd3e9137dc/users/cc0fc06b83cd42b490e47f8b3d8498c1/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:03.934097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a098aacb8cc147ad8b22c3dd3e9137dc, user_id=cc0fc06b83cd42b490e47f8b3d8498c1, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:03.936958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:04.013911 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1292/2611] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:03 2022] PUT /identity/v3/projects/a098aacb8cc147ad8b22c3dd3e9137dc/users/cc0fc06b83cd42b490e47f8b3d8498c1/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:04.026052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1dd42-a243-4ee4-91ef-a239c4dcb26d None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:04.026455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1dd42-a243-4ee4-91ef-a239c4dcb26d None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:04.026947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1dd42-a243-4ee4-91ef-a239c4dcb26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:04.057565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-8ce1dd42-a243-4ee4-91ef-a239c4dcb26d None None] MFA Rules not processed for user `cc0fc06b83cd42b490e47f8b3d8498c1`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:04.058929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-8ce1dd42-a243-4ee4-91ef-a239c4dcb26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:04.125773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:04.128538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:04.144276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1320/2612] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:04.146987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1dd42-a243-4ee4-91ef-a239c4dcb26d None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:04.203361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e966eb-58d7-440f-8601-bcc28d676d19 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:04.208769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:04.209826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:04.210113 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:04.210398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] PATH_INFO: `/v3/users/28df35ff2d2d4d85b3990edee18232da` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:04.211625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28df35ff2d2d4d85b3990edee18232da)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:04.213616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:04.272043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1dd42-a243-4ee4-91ef-a239c4dcb26d None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:04.278871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:04.280218 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:04.280560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:04.280930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:04.281710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:04.286990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:04.427012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1321/2613] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:04 2022] POST /identity/v3/projects => generated 367 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:04.427012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:04.428777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:04.488697 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28df35ff2d2d4d85b3990edee18232da'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:04.510764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a569ff73-b595-497e-b7ca-13ea681af56f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:04.517317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:04.518211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:04.518490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:04.519318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:04.524391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:04.604503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28df35ff2d2d4d85b3990edee18232da'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:04.605127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28df35ff2d2d4d85b3990edee18232da'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:04.607723 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:04.613312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:04.617128 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28df35ff2d2d4d85b3990edee18232da'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:04.618657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-9c5a1a59-4231-482c-bcb5-375bd01db378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28df35ff2d2d4d85b3990edee18232da'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:04.634046 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1322/2614] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:04.635265 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:04.661881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1293/2615] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:04 2022] DELETE /identity/v3/users/28df35ff2d2d4d85b3990edee18232da => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:04.665333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd705f-45df-496f-bea3-c897f7d8e0dd service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:04.667011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443281d5-344e-4f38-b7c2-b633b720169d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:04.668468 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443281d5-344e-4f38-b7c2-b633b720169d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:04.668955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443281d5-344e-4f38-b7c2-b633b720169d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:04.693520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-443281d5-344e-4f38-b7c2-b633b720169d None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:04.695123 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-443281d5-344e-4f38-b7c2-b633b720169d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:04.797270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:04.798977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:04.802630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:04.803163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:04.803957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:04.806846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:04.811668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:04.833585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:04.843474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1294/2616] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:04.858011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443281d5-344e-4f38-b7c2-b633b720169d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:04.860096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-443281d5-344e-4f38-b7c2-b633b720169d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:04.931076 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1323/2617] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:04 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:04.935820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:04.945197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:04.979069 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15a3a9-2213-4b37-8d59-1f1a339ad152 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:04.988984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443281d5-344e-4f38-b7c2-b633b720169d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:04.993123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe3d59-3f1c-40bb-bca2-58e31e1c450c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:04.994017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df0ba8-3477-404d-8f7c-58cabc59c4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:04.994345 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe3d59-3f1c-40bb-bca2-58e31e1c450c admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:04.994633 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe3d59-3f1c-40bb-bca2-58e31e1c450c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:04.994920 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe3d59-3f1c-40bb-bca2-58e31e1c450c admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:04.995044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0ba8-3477-404d-8f7c-58cabc59c4cb service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:04.995432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0ba8-3477-404d-8f7c-58cabc59c4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:04.995807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0ba8-3477-404d-8f7c-58cabc59c4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:04.997234 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe3d59-3f1c-40bb-bca2-58e31e1c450c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:05.002695 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe3d59-3f1c-40bb-bca2-58e31e1c450c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:05.008338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df0ba8-3477-404d-8f7c-58cabc59c4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:05.010287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df0ba8-3477-404d-8f7c-58cabc59c4cb service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:05.025230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1295/2618] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:07:05.034880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1324/2619] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:05.042163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df0ba8-3477-404d-8f7c-58cabc59c4cb service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:05.062653 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df0ba8-3477-404d-8f7c-58cabc59c4cb service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:05.068738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b3fe6-fbcd-41f7-b0a9-2e0064589193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:05.069836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3fe6-fbcd-41f7-b0a9-2e0064589193 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:05.070359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3fe6-fbcd-41f7-b0a9-2e0064589193 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:05.070738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b3fe6-fbcd-41f7-b0a9-2e0064589193 admin admin] PATH_INFO: `/v3/projects/c97d1336658b4200af6817ccc1056ee3/users/a7be5909ae564627b07ed1549251438e/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:05.089397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b3fe6-fbcd-41f7-b0a9-2e0064589193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c97d1336658b4200af6817ccc1056ee3, user_id=a7be5909ae564627b07ed1549251438e, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:05.092762 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b3fe6-fbcd-41f7-b0a9-2e0064589193 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:05.201725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1296/2620] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:05 2022] PUT /identity/v3/projects/c97d1336658b4200af6817ccc1056ee3/users/a7be5909ae564627b07ed1549251438e/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:05.205850 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe3d59-3f1c-40bb-bca2-58e31e1c450c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:05.216871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe3d59-3f1c-40bb-bca2-58e31e1c450c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:05.288680 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f074e7-3ddd-46f7-861d-6f7a7e8869e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:05.289351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f074e7-3ddd-46f7-861d-6f7a7e8869e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:05.289637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f074e7-3ddd-46f7-861d-6f7a7e8869e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:05.289984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f074e7-3ddd-46f7-861d-6f7a7e8869e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:05.290854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f074e7-3ddd-46f7-861d-6f7a7e8869e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:05.292588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f074e7-3ddd-46f7-861d-6f7a7e8869e7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:05.306970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1325/2621] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:05.312504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b3fe6-fbcd-41f7-b0a9-2e0064589193 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:05.335480 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc4583-06e1-4dfb-a42d-2238d604ec2b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:05.336007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc4583-06e1-4dfb-a42d-2238d604ec2b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:05.336450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc4583-06e1-4dfb-a42d-2238d604ec2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:05.339546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b3fe6-fbcd-41f7-b0a9-2e0064589193 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:05.345899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871a8ef-c116-44ca-a200-e80a0bb06e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:05.347121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871a8ef-c116-44ca-a200-e80a0bb06e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:05.347636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871a8ef-c116-44ca-a200-e80a0bb06e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:05.348166 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871a8ef-c116-44ca-a200-e80a0bb06e98 admin admin] PATH_INFO: `/v3/projects/c97d1336658b4200af6817ccc1056ee3/users/a7be5909ae564627b07ed1549251438e/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:05.351125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871a8ef-c116-44ca-a200-e80a0bb06e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c97d1336658b4200af6817ccc1056ee3, user_id=a7be5909ae564627b07ed1549251438e, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:05.355105 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871a8ef-c116-44ca-a200-e80a0bb06e98 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:05.366969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:05.370450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-37bc4583-06e1-4dfb-a42d-2238d604ec2b None None] MFA Rules not processed for user `ee69b3f14a814f0aaa4de380bf2cc819`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:05.371716 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-37bc4583-06e1-4dfb-a42d-2238d604ec2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:05.390972 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1297/2622] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:05 2022] PUT /identity/v3/projects/c97d1336658b4200af6817ccc1056ee3/users/a7be5909ae564627b07ed1549251438e/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:05.392443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48c6dc-47b8-443e-a136-a283e6814d1a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:05.392763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48c6dc-47b8-443e-a136-a283e6814d1a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:05.393020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48c6dc-47b8-443e-a136-a283e6814d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:05.425024 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ba48c6dc-47b8-443e-a136-a283e6814d1a None None] MFA Rules not processed for user `a7be5909ae564627b07ed1549251438e`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:05.426481 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ba48c6dc-47b8-443e-a136-a283e6814d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:05.500858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1326/2623] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:05.511529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc4583-06e1-4dfb-a42d-2238d604ec2b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:05.513751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-37bc4583-06e1-4dfb-a42d-2238d604ec2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:05.540162 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1298/2624] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:05.551967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48c6dc-47b8-443e-a136-a283e6814d1a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:05.555269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ba48c6dc-47b8-443e-a136-a283e6814d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:05.562405 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:05.625088 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc4583-06e1-4dfb-a42d-2238d604ec2b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:05.627663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48c6dc-47b8-443e-a136-a283e6814d1a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:05.632022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b3024-34c0-48c7-8784-471b3811083e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:05.632502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:05.633296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:05.633433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b3024-34c0-48c7-8784-471b3811083e service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:05.633583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b3024-34c0-48c7-8784-471b3811083e service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:05.633869 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b3024-34c0-48c7-8784-471b3811083e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:05.634111 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:05.634317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:05.636421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:05.645107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b3024-34c0-48c7-8784-471b3811083e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:05.650582 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b3024-34c0-48c7-8784-471b3811083e service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:05.662586 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1299/2625] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:05.731204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=100 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:05.739270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:05.741015 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:05.772511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1327/2626] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:07:05.945224 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b3024-34c0-48c7-8784-471b3811083e service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:05.955273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b3024-34c0-48c7-8784-471b3811083e service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:05.960524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:05.961276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:05.961636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:05.961924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:05.964086 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:06.033289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:06.034778 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:06.047666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1300/2627] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:06.052598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:06.056826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:06.172979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362be9d-4276-4298-b3fe-459c6dd1b7a4 service nova] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:06.183431 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:06.184747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:06.185211 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:06.186169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] PATH_INFO: `/v3/projects/79f27d76abb74f8bbbae75a49e153526` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:06.189256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f27d76abb74f8bbbae75a49e153526)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:06.191456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:06.338259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f27d76abb74f8bbbae75a49e153526', 'request_id': 'req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.421075 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f27d76abb74f8bbbae75a49e153526', 'request_id': 'req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.434328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f27d76abb74f8bbbae75a49e153526', 'request_id': 'req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.452903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] The token cache is being invalidate because project 79f27d76abb74f8bbbae75a49e153526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:06.453219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.453883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.455782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1328/2628] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:06 2022] DELETE /identity/v3/projects/79f27d76abb74f8bbbae75a49e153526 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:06.459487 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:06.487069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e58cfe-5fa0-491c-a6dc-0fba127822d8 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:06.633243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:06.636918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:06.636918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:06.636918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] PATH_INFO: `/v3/users/98b9e6be6148465d823a0f40a94c3607` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:06.636918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b9e6be6148465d823a0f40a94c3607)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:06.638831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:06.835325 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b9e6be6148465d823a0f40a94c3607'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.837073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:06.914840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b9e6be6148465d823a0f40a94c3607'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.915482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b9e6be6148465d823a0f40a94c3607'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.927936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b9e6be6148465d823a0f40a94c3607'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.929282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4b58569e-529c-41e9-9d9b-24d5001b87a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b9e6be6148465d823a0f40a94c3607'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:06.957398 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1301/2629] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:06 2022] DELETE /identity/v3/users/98b9e6be6148465d823a0f40a94c3607 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:07.580281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:07.581845 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:07.595848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fd65-6dc4-4369-b32d-b6a8a4b98773 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:07.630456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:07.680555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:07.681445 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:07.681946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:07.682447 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] PATH_INFO: `/v3/projects/4a366549bfd24e11b503c01eba1225ff` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:07.683976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a366549bfd24e11b503c01eba1225ff)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:07.687376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:07.823775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a366549bfd24e11b503c01eba1225ff', 'request_id': 'req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:07.891456 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a366549bfd24e11b503c01eba1225ff', 'request_id': 'req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:07.898291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a366549bfd24e11b503c01eba1225ff', 'request_id': 'req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:07.911181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] The token cache is being invalidate because project 4a366549bfd24e11b503c01eba1225ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:07.911475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:07.916670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-7cfeaf32-ba37-4ac3-a9a9-e74401a17698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:07.921181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1329/2630] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:07 2022] DELETE /identity/v3/projects/4a366549bfd24e11b503c01eba1225ff => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:07.926110 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87da2c0-1db1-4f56-b594-7f3543f11a7b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:07.926896 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87da2c0-1db1-4f56-b594-7f3543f11a7b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:07.927564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87da2c0-1db1-4f56-b594-7f3543f11a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:07.950621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f87da2c0-1db1-4f56-b594-7f3543f11a7b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:07.952644 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f87da2c0-1db1-4f56-b594-7f3543f11a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:08.022375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1302/2631] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:08.031178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b35c18-f4e2-47b9-a942-fce1885bd9e4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:08.031467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b35c18-f4e2-47b9-a942-fce1885bd9e4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:08.031799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b35c18-f4e2-47b9-a942-fce1885bd9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:08.054137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-29b35c18-f4e2-47b9-a942-fce1885bd9e4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:08.055406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-29b35c18-f4e2-47b9-a942-fce1885bd9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:08.065951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1330/2632] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:08.070584 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87da2c0-1db1-4f56-b594-7f3543f11a7b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:08.082781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87da2c0-1db1-4f56-b594-7f3543f11a7b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:08.088362 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:08.089033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:08.089605 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:08.090051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:08.090825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:08.091293 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:08.093148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:08.106770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1303/2633] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:07:08.111859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b35c18-f4e2-47b9-a942-fce1885bd9e4 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:08.121256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b35c18-f4e2-47b9-a942-fce1885bd9e4 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:08.126997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a6a6-91d9-4085-a529-338e594812e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:08.128031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a6a6-91d9-4085-a529-338e594812e7 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:08.128031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a6a6-91d9-4085-a529-338e594812e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:08.128261 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59a6a6-91d9-4085-a529-338e594812e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:08.128908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59a6a6-91d9-4085-a529-338e594812e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:08.130837 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59a6a6-91d9-4085-a529-338e594812e7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:08.353509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1331/2634] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:08 2022] POST /identity/v3/projects => generated 359 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:08.354537 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:08.365353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e413d7-5410-4f6c-aae4-727c9753ce64 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:08.449715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8ebda-18a3-405f-b433-b1de3b4f162a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:08.450148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8ebda-18a3-405f-b433-b1de3b4f162a admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:08.450555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8ebda-18a3-405f-b433-b1de3b4f162a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:08.450988 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8ebda-18a3-405f-b433-b1de3b4f162a admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:08.452270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8ebda-18a3-405f-b433-b1de3b4f162a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:08.454394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8ebda-18a3-405f-b433-b1de3b4f162a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:08.527892 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1304/2635] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:08 2022] POST /identity/v3/users => generated 280 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:07:08.532357 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a6a6-91d9-4085-a529-338e594812e7 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:08.547304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59a6a6-91d9-4085-a529-338e594812e7 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:08.555988 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb23b-6d0e-4476-9a9b-e66a87a21660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:08.557374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb23b-6d0e-4476-9a9b-e66a87a21660 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:08.557703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb23b-6d0e-4476-9a9b-e66a87a21660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:08.558199 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbb23b-6d0e-4476-9a9b-e66a87a21660 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:08.559041 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbb23b-6d0e-4476-9a9b-e66a87a21660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:08.561335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbb23b-6d0e-4476-9a9b-e66a87a21660 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:08.572751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1332/2636] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:08.576872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8ebda-18a3-405f-b433-b1de3b4f162a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:08.600009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8ebda-18a3-405f-b433-b1de3b4f162a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:08.610523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46adafaf-54b3-4844-8378-ad3eaf8a3087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:08.614400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adafaf-54b3-4844-8378-ad3eaf8a3087 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:08.614920 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adafaf-54b3-4844-8378-ad3eaf8a3087 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:08.615338 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46adafaf-54b3-4844-8378-ad3eaf8a3087 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:08.616210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adafaf-54b3-4844-8378-ad3eaf8a3087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:08.620048 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46adafaf-54b3-4844-8378-ad3eaf8a3087 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:08.654764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1305/2637] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:08.658836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb23b-6d0e-4476-9a9b-e66a87a21660 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:08.678465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbb23b-6d0e-4476-9a9b-e66a87a21660 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:08.683300 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:08.684653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:08.685429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:08.686104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] PATH_INFO: `/v3/projects/ed8761b9d27f445f86b0f93cf92ac137/users/1a04d706f57e483db76b60a3b28fd719/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:08.708798 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed8761b9d27f445f86b0f93cf92ac137, user_id=1a04d706f57e483db76b60a3b28fd719, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:08.716651 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:08.808692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1333/2638] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:08 2022] PUT /identity/v3/projects/ed8761b9d27f445f86b0f93cf92ac137/users/1a04d706f57e483db76b60a3b28fd719/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:08.822261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd621-dbe7-46ac-92e2-dc00c4f53dca None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:08.822903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd621-dbe7-46ac-92e2-dc00c4f53dca None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:08.823324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973bd621-dbe7-46ac-92e2-dc00c4f53dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:08.856178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-973bd621-dbe7-46ac-92e2-dc00c4f53dca None None] MFA Rules not processed for user `1a04d706f57e483db76b60a3b28fd719`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:08.857619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-973bd621-dbe7-46ac-92e2-dc00c4f53dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:08.877205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:08.961704 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1306/2639] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:08 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:09.004801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:09.006710 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:09.091112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca630a8-5078-49b4-b7a3-d555e693ec83 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:09.097832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:09.098751 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:09.099354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:09.099812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:09.120954 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:09.122666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:09.135471 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1334/2640] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:09 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:09.151529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bd621-dbe7-46ac-92e2-dc00c4f53dca None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:09.239236 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973bd621-dbe7-46ac-92e2-dc00c4f53dca None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:09.245756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:09.246957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:09.247473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:09.248505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] PATH_INFO: `/v3/users/1a04d706f57e483db76b60a3b28fd719` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:09.249960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a04d706f57e483db76b60a3b28fd719)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:09.252519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:09.631873 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a04d706f57e483db76b60a3b28fd719'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:09.735562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a04d706f57e483db76b60a3b28fd719'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:09.736315 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a04d706f57e483db76b60a3b28fd719'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:09.744508 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a04d706f57e483db76b60a3b28fd719'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:09.745485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-20b3427f-da10-4e96-a191-58c976c1a33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a04d706f57e483db76b60a3b28fd719'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:09.793873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:09.798019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:09.822707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:09.825239 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1307/2641] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:09 2022] DELETE /identity/v3/users/1a04d706f57e483db76b60a3b28fd719 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:09.827067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893db16a-0a68-4785-946e-433661ccc685 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:09.829321 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023bf7-3d03-441e-87cc-a6eeb76b56b5 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:09.829616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023bf7-3d03-441e-87cc-a6eeb76b56b5 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:09.829903 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023bf7-3d03-441e-87cc-a6eeb76b56b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:09.862924 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-a3023bf7-3d03-441e-87cc-a6eeb76b56b5 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:09.864185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a3023bf7-3d03-441e-87cc-a6eeb76b56b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:09.906473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:09.911461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:09.912626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:09.912913 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:09.917198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] PATH_INFO: `/v3/users/d485f9fc38a442dd9e1528024460fcca` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:09.917198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d485f9fc38a442dd9e1528024460fcca)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:09.917838 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:09.942854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1308/2642] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:09.953757 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3023bf7-3d03-441e-87cc-a6eeb76b56b5 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:09.955602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a3023bf7-3d03-441e-87cc-a6eeb76b56b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:10.018705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3023bf7-3d03-441e-87cc-a6eeb76b56b5 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:10.022850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:10.023618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:10.023974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:10.024597 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:10.035151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:10.036727 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:10.048868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1309/2643] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:07:10.167128 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd485f9fc38a442dd9e1528024460fcca'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:10.269879 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd485f9fc38a442dd9e1528024460fcca'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:10.270997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd485f9fc38a442dd9e1528024460fcca'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:10.280704 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd485f9fc38a442dd9e1528024460fcca'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:10.281450 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd485f9fc38a442dd9e1528024460fcca'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:10.302618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1335/2644] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:09 2022] DELETE /identity/v3/users/d485f9fc38a442dd9e1528024460fcca => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:10.579883 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:10.581472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:10.602655 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafc99d-9d5c-4f51-8980-e3c6756dfe66 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:10.657254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:10.658724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:10.731300 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:10.732939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:10.733463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:10.734089 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] PATH_INFO: `/v3/projects/ed8761b9d27f445f86b0f93cf92ac137` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:10.735544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed8761b9d27f445f86b0f93cf92ac137)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:10.740380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:10.775858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9faba9-d2fa-4a01-80ce-5e2ed030a1b1 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:10.781692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:10.782565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:10.782844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:10.783120 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] PATH_INFO: `/v3/projects/1871c9af781743f5af1d92371a42ecf1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:10.784328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1871c9af781743f5af1d92371a42ecf1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:10.786276 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:10.907503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed8761b9d27f445f86b0f93cf92ac137', 'request_id': 'req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:10.933923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1871c9af781743f5af1d92371a42ecf1', 'request_id': 'req-525d158d-7e96-4856-9994-0a20988fbe0d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.016503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed8761b9d27f445f86b0f93cf92ac137', 'request_id': 'req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.026854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed8761b9d27f445f86b0f93cf92ac137', 'request_id': 'req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.048074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] The token cache is being invalidate because project ed8761b9d27f445f86b0f93cf92ac137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:11.048397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.049848 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4d33c9b5-9e2d-4a22-8840-fb2ef75a32c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.055496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1310/2645] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:10 2022] DELETE /identity/v3/projects/ed8761b9d27f445f86b0f93cf92ac137 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:11.058027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc26dd-bae8-447c-96ae-1999033f5601 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.058485 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc26dd-bae8-447c-96ae-1999033f5601 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.058951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc26dd-bae8-447c-96ae-1999033f5601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.083093 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e6bc26dd-bae8-447c-96ae-1999033f5601 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:11.084630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e6bc26dd-bae8-447c-96ae-1999033f5601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:11.098661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1871c9af781743f5af1d92371a42ecf1', 'request_id': 'req-525d158d-7e96-4856-9994-0a20988fbe0d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.107868 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1871c9af781743f5af1d92371a42ecf1', 'request_id': 'req-525d158d-7e96-4856-9994-0a20988fbe0d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.152394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] The token cache is being invalidate because project 1871c9af781743f5af1d92371a42ecf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:11.153409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.153672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1311/2646] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:11.159422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-525d158d-7e96-4856-9994-0a20988fbe0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:11.160860 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1336/2647] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:10 2022] DELETE /identity/v3/projects/1871c9af781743f5af1d92371a42ecf1 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:11.162529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01278d0b-4ba5-4748-8c26-1348d7e8778a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.162818 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01278d0b-4ba5-4748-8c26-1348d7e8778a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.163098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01278d0b-4ba5-4748-8c26-1348d7e8778a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.165348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761c3ab-3b92-45d3-b4f0-27c58f7cab87 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.165635 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761c3ab-3b92-45d3-b4f0-27c58f7cab87 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.165930 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761c3ab-3b92-45d3-b4f0-27c58f7cab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.187140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d761c3ab-3b92-45d3-b4f0-27c58f7cab87 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:11.188513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d761c3ab-3b92-45d3-b4f0-27c58f7cab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:11.197082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-01278d0b-4ba5-4748-8c26-1348d7e8778a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:11.197616 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1337/2648] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:11.198395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-01278d0b-4ba5-4748-8c26-1348d7e8778a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:11.206237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2104a-fb7e-4fd1-a85d-3677d6578829 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.206537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2104a-fb7e-4fd1-a85d-3677d6578829 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.214620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c2104a-fb7e-4fd1-a85d-3677d6578829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.214763 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1312/2649] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:11.218374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01278d0b-4ba5-4748-8c26-1348d7e8778a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.226417 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-82c2104a-fb7e-4fd1-a85d-3677d6578829 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:11.227620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-82c2104a-fb7e-4fd1-a85d-3677d6578829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:11.230136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01278d0b-4ba5-4748-8c26-1348d7e8778a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.239039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.239795 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.241289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.241574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.242475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:11.242791 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.244699 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.250749 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1338/2650] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:11.251736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2104a-fb7e-4fd1-a85d-3677d6578829 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.261617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c2104a-fb7e-4fd1-a85d-3677d6578829 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.266160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1313/2651] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:07:11.267309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.267459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.268294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.275619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.275619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.275619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:11.275619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.275619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.280518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15397e-63f2-442c-bd54-d3d600ca1e15 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.285854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c407bd-622c-4fa9-b4eb-14dd14067478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.288034 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c407bd-622c-4fa9-b4eb-14dd14067478 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.288477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c407bd-622c-4fa9-b4eb-14dd14067478 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.289088 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c407bd-622c-4fa9-b4eb-14dd14067478 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.289833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1339/2652] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:07:11.290399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c407bd-622c-4fa9-b4eb-14dd14067478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.290783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.296163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c407bd-622c-4fa9-b4eb-14dd14067478 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.307035 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef8c53-3f04-4018-86e2-5cbfd74cb6eb admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.317123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4af81b-0398-45a7-af85-560805d04cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.317647 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4af81b-0398-45a7-af85-560805d04cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.317924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4af81b-0398-45a7-af85-560805d04cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.318206 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4af81b-0398-45a7-af85-560805d04cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.318914 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4af81b-0398-45a7-af85-560805d04cf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.321896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4af81b-0398-45a7-af85-560805d04cf9 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.424355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1314/2653] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:11.429144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c407bd-622c-4fa9-b4eb-14dd14067478 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.449304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1340/2654] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:11.454387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4af81b-0398-45a7-af85-560805d04cf9 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.514922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c407bd-622c-4fa9-b4eb-14dd14067478 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.560805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4af81b-0398-45a7-af85-560805d04cf9 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.577022 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfded-932e-4c27-b434-e2d4eae0a868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.578330 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfded-932e-4c27-b434-e2d4eae0a868 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.578774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfded-932e-4c27-b434-e2d4eae0a868 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.579212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bfded-932e-4c27-b434-e2d4eae0a868 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.580142 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bfded-932e-4c27-b434-e2d4eae0a868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.582795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bfded-932e-4c27-b434-e2d4eae0a868 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.587998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:11.606826 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5d5b-c389-4ac2-8c90-70a887a420be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.608938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5d5b-c389-4ac2-8c90-70a887a420be admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.609326 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5d5b-c389-4ac2-8c90-70a887a420be admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.609744 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa5d5b-c389-4ac2-8c90-70a887a420be admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.610551 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa5d5b-c389-4ac2-8c90-70a887a420be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.612819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa5d5b-c389-4ac2-8c90-70a887a420be admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.673839 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1341/2655] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:11.675489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfded-932e-4c27-b434-e2d4eae0a868 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.693274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bfded-932e-4c27-b434-e2d4eae0a868 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.693989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1315/2656] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:11.697828 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5d5b-c389-4ac2-8c90-70a887a420be admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.699318 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5379ad1-49c3-424b-9553-a74d4a1319e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.700002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379ad1-49c3-424b-9553-a74d4a1319e7 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.700475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379ad1-49c3-424b-9553-a74d4a1319e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.700762 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5379ad1-49c3-424b-9553-a74d4a1319e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.705889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5379ad1-49c3-424b-9553-a74d4a1319e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.708224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5379ad1-49c3-424b-9553-a74d4a1319e7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.721075 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa5d5b-c389-4ac2-8c90-70a887a420be admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.729110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1342/2657] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:11.733878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5379ad1-49c3-424b-9553-a74d4a1319e7 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.738216 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d2c71-7e0b-41f7-830b-758dc65bf92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.739131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d2c71-7e0b-41f7-830b-758dc65bf92c admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.739555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d2c71-7e0b-41f7-830b-758dc65bf92c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.739958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d2c71-7e0b-41f7-830b-758dc65bf92c admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.740865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d2c71-7e0b-41f7-830b-758dc65bf92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.743625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d2c71-7e0b-41f7-830b-758dc65bf92c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.761167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1316/2658] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:11.766438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d2c71-7e0b-41f7-830b-758dc65bf92c admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.768795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5379ad1-49c3-424b-9553-a74d4a1319e7 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.782502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d2c71-7e0b-41f7-830b-758dc65bf92c admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.782804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8bf0f-df0b-4c5c-83af-9e264b51c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.783585 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8bf0f-df0b-4c5c-83af-9e264b51c360 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.783949 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8bf0f-df0b-4c5c-83af-9e264b51c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.784331 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8bf0f-df0b-4c5c-83af-9e264b51c360 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.785294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8bf0f-df0b-4c5c-83af-9e264b51c360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.785479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:11.787216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8bf0f-df0b-4c5c-83af-9e264b51c360 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.790611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab1d09-d279-4156-9480-d7b8566905e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.792566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab1d09-d279-4156-9480-d7b8566905e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.793043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab1d09-d279-4156-9480-d7b8566905e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.793499 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab1d09-d279-4156-9480-d7b8566905e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.794431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab1d09-d279-4156-9480-d7b8566905e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.796741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab1d09-d279-4156-9480-d7b8566905e2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.806049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1343/2659] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:11.813395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8bf0f-df0b-4c5c-83af-9e264b51c360 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.820225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1317/2660] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:11.824211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab1d09-d279-4156-9480-d7b8566905e2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:11.828272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8bf0f-df0b-4c5c-83af-9e264b51c360 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.833956 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f35467-7a09-41d8-b054-2d6b0ae3c13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.835101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f35467-7a09-41d8-b054-2d6b0ae3c13b admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.835253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab1d09-d279-4156-9480-d7b8566905e2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:11.835509 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f35467-7a09-41d8-b054-2d6b0ae3c13b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.835855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f35467-7a09-41d8-b054-2d6b0ae3c13b admin admin] PATH_INFO: `/v3/projects/a6d19da839b9448aa7c04b0b2cdeb30f/users/ab0ba257489c491ca4cd1b64d8e456a1/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.840410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad63235-093e-40e1-b36a-e4882558ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:11.841467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad63235-093e-40e1-b36a-e4882558ef05 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.842636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad63235-093e-40e1-b36a-e4882558ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.843045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad63235-093e-40e1-b36a-e4882558ef05 admin admin] PATH_INFO: `/v3/projects/54ac4601e8a747698cc21d2e99b56f8f/users/2d4130e9f92e41dd9118f6080098cd36/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.859063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f35467-7a09-41d8-b054-2d6b0ae3c13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6d19da839b9448aa7c04b0b2cdeb30f, user_id=ab0ba257489c491ca4cd1b64d8e456a1, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.859815 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad63235-093e-40e1-b36a-e4882558ef05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ac4601e8a747698cc21d2e99b56f8f, user_id=2d4130e9f92e41dd9118f6080098cd36, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:11.861910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f35467-7a09-41d8-b054-2d6b0ae3c13b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.862538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad63235-093e-40e1-b36a-e4882558ef05 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:11.960693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1344/2661] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:11 2022] PUT /identity/v3/projects/a6d19da839b9448aa7c04b0b2cdeb30f/users/ab0ba257489c491ca4cd1b64d8e456a1/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:11.971892 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4b59f-3c90-4b08-b3d8-fb88ab1f7ae5 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:11.972245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4b59f-3c90-4b08-b3d8-fb88ab1f7ae5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:11.972575 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4b59f-3c90-4b08-b3d8-fb88ab1f7ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:11.995025 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1318/2662] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:11 2022] PUT /identity/v3/projects/54ac4601e8a747698cc21d2e99b56f8f/users/2d4130e9f92e41dd9118f6080098cd36/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:12.001793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-edd4b59f-3c90-4b08-b3d8-fb88ab1f7ae5 None None] MFA Rules not processed for user `ab0ba257489c491ca4cd1b64d8e456a1`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:12.003022 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-edd4b59f-3c90-4b08-b3d8-fb88ab1f7ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:12.006717 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4a020-e451-41ec-9697-6a5ec7ece418 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.007017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4a020-e451-41ec-9697-6a5ec7ece418 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.007535 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4a020-e451-41ec-9697-6a5ec7ece418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.045411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-06d4a020-e451-41ec-9697-6a5ec7ece418 None None] MFA Rules not processed for user `2d4130e9f92e41dd9118f6080098cd36`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:12.047523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-06d4a020-e451-41ec-9697-6a5ec7ece418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:12.092002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1345/2663] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:12.097096 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4b59f-3c90-4b08-b3d8-fb88ab1f7ae5 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:12.128877 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1319/2664] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:12.146152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4a020-e451-41ec-9697-6a5ec7ece418 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:12.150122 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-06d4a020-e451-41ec-9697-6a5ec7ece418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:12.184614 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd4b59f-3c90-4b08-b3d8-fb88ab1f7ae5 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:12.190811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365f4d0-ff4e-4eba-9f32-dc1bc44bc072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:12.191290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f4d0-ff4e-4eba-9f32-dc1bc44bc072 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.191512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f4d0-ff4e-4eba-9f32-dc1bc44bc072 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.191789 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365f4d0-ff4e-4eba-9f32-dc1bc44bc072 admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.192639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365f4d0-ff4e-4eba-9f32-dc1bc44bc072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:12.194611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365f4d0-ff4e-4eba-9f32-dc1bc44bc072 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:12.214542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4a020-e451-41ec-9697-6a5ec7ece418 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:12.218502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d740bf-412b-4446-9020-d16415ecad38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:12.219152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d740bf-412b-4446-9020-d16415ecad38 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.219437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d740bf-412b-4446-9020-d16415ecad38 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.219711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d740bf-412b-4446-9020-d16415ecad38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.228712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d740bf-412b-4446-9020-d16415ecad38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:12.230127 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d740bf-412b-4446-9020-d16415ecad38 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:12.239563 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1320/2665] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:12.325736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1346/2666] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:12 2022] POST /identity/v3/projects => generated 373 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:12.328043 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d740bf-412b-4446-9020-d16415ecad38 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:12.339328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d740bf-412b-4446-9020-d16415ecad38 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:12.398276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0c25d-7113-4e5a-bd97-4bd479d89db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:12.399039 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0c25d-7113-4e5a-bd97-4bd479d89db6 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.399543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0c25d-7113-4e5a-bd97-4bd479d89db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.399870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0c25d-7113-4e5a-bd97-4bd479d89db6 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.400613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0c25d-7113-4e5a-bd97-4bd479d89db6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:12.402541 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0c25d-7113-4e5a-bd97-4bd479d89db6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:12.519638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1321/2667] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:12 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:12.520715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365f4d0-ff4e-4eba-9f32-dc1bc44bc072 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:12.533587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365f4d0-ff4e-4eba-9f32-dc1bc44bc072 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:12.549393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e8ef-f3b4-48dc-a90f-f6d09bba9fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:12.550306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e8ef-f3b4-48dc-a90f-f6d09bba9fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.550657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e8ef-f3b4-48dc-a90f-f6d09bba9fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.551010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e8ef-f3b4-48dc-a90f-f6d09bba9fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.551948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2e8ef-f3b4-48dc-a90f-f6d09bba9fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:12.553724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2e8ef-f3b4-48dc-a90f-f6d09bba9fa7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:12.568701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1347/2668] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:12.573292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0c25d-7113-4e5a-bd97-4bd479d89db6 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:12.584746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0c25d-7113-4e5a-bd97-4bd479d89db6 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:12.590556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda0678-6c7e-4de9-8233-b7963954cd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:12.592027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda0678-6c7e-4de9-8233-b7963954cd2e admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.592467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda0678-6c7e-4de9-8233-b7963954cd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.593179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda0678-6c7e-4de9-8233-b7963954cd2e admin admin] PATH_INFO: `/v3/projects/e005de374bbd455cb6ec25f7c48c5f6e/users/59415bfd1a8e4d6c8b846ae018af6689/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.615554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda0678-6c7e-4de9-8233-b7963954cd2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e005de374bbd455cb6ec25f7c48c5f6e, user_id=59415bfd1a8e4d6c8b846ae018af6689, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:12.618383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda0678-6c7e-4de9-8233-b7963954cd2e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:12.693185 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1322/2669] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:12 2022] PUT /identity/v3/projects/e005de374bbd455cb6ec25f7c48c5f6e/users/59415bfd1a8e4d6c8b846ae018af6689/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:12.698064 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e8ef-f3b4-48dc-a90f-f6d09bba9fa7 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:12.714957 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e8ef-f3b4-48dc-a90f-f6d09bba9fa7 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:12.771825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a8a6b-a842-4d06-9fde-36d35498d6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:12.772593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a8a6b-a842-4d06-9fde-36d35498d6cb admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.772880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a8a6b-a842-4d06-9fde-36d35498d6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.773584 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a8a6b-a842-4d06-9fde-36d35498d6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.773908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a8a6b-a842-4d06-9fde-36d35498d6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:12.775801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a8a6b-a842-4d06-9fde-36d35498d6cb admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:12.786847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1348/2670] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:12.791465 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda0678-6c7e-4de9-8233-b7963954cd2e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:12.802595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda0678-6c7e-4de9-8233-b7963954cd2e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:12.807521 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d5a3e-2fb8-4bd7-850c-3b6be3486d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:12.808455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5a3e-2fb8-4bd7-850c-3b6be3486d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.808738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5a3e-2fb8-4bd7-850c-3b6be3486d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.809010 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5a3e-2fb8-4bd7-850c-3b6be3486d54 admin admin] PATH_INFO: `/v3/projects/e005de374bbd455cb6ec25f7c48c5f6e/users/59415bfd1a8e4d6c8b846ae018af6689/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.811760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d5a3e-2fb8-4bd7-850c-3b6be3486d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e005de374bbd455cb6ec25f7c48c5f6e, user_id=59415bfd1a8e4d6c8b846ae018af6689, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:12.814467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d5a3e-2fb8-4bd7-850c-3b6be3486d54 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:12.825115 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1323/2671] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:12 2022] PUT /identity/v3/projects/e005de374bbd455cb6ec25f7c48c5f6e/users/59415bfd1a8e4d6c8b846ae018af6689/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:12.836282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7918e-b53c-4691-a4bb-5c2906ba957b None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:12.836576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7918e-b53c-4691-a4bb-5c2906ba957b None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:12.836859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7918e-b53c-4691-a4bb-5c2906ba957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:12.845178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:12.867249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-69f7918e-b53c-4691-a4bb-5c2906ba957b None None] MFA Rules not processed for user `59415bfd1a8e4d6c8b846ae018af6689`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:12.868502 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-69f7918e-b53c-4691-a4bb-5c2906ba957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:12.944298 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1349/2672] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:12.955621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d5a3e-2fb8-4bd7-850c-3b6be3486d54 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:12.989375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:13.021524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d5a3e-2fb8-4bd7-850c-3b6be3486d54 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:13.025413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61325a96-4abc-4aec-8200-51f7fc764f4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:13.026063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61325a96-4abc-4aec-8200-51f7fc764f4b service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:13.026350 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61325a96-4abc-4aec-8200-51f7fc764f4b service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:13.026627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61325a96-4abc-4aec-8200-51f7fc764f4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:13.036642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61325a96-4abc-4aec-8200-51f7fc764f4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:13.038080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61325a96-4abc-4aec-8200-51f7fc764f4b service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:13.048684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1324/2673] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:13.494893 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7918e-b53c-4691-a4bb-5c2906ba957b None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:13.496351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-69f7918e-b53c-4691-a4bb-5c2906ba957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:13.563159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7918e-b53c-4691-a4bb-5c2906ba957b None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:13.568123 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:13.569025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:13.569308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:13.569618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] PATH_INFO: `/v3/users/cc0fc06b83cd42b490e47f8b3d8498c1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:13.570882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc0fc06b83cd42b490e47f8b3d8498c1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:13.572901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:13.657773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:13.778966 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc0fc06b83cd42b490e47f8b3d8498c1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:13.834903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc0fc06b83cd42b490e47f8b3d8498c1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:13.835538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc0fc06b83cd42b490e47f8b3d8498c1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:13.842772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc0fc06b83cd42b490e47f8b3d8498c1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:13.843857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc0fc06b83cd42b490e47f8b3d8498c1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:13.860087 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1350/2674] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:13 2022] DELETE /identity/v3/users/cc0fc06b83cd42b490e47f8b3d8498c1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:13.863876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:13.864587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c0e78-8f9d-4506-a44f-f3304547cdb4 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:13.864876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c0e78-8f9d-4506-a44f-f3304547cdb4 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:13.865150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c0e78-8f9d-4506-a44f-f3304547cdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:13.883591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f34c0e78-8f9d-4506-a44f-f3304547cdb4 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:13.884859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f34c0e78-8f9d-4506-a44f-f3304547cdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:13.914666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:13.916377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:13.947574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1325/2675] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:13.959056 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c0e78-8f9d-4506-a44f-f3304547cdb4 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:13.961505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f34c0e78-8f9d-4506-a44f-f3304547cdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:13.988342 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9975e5a-c56f-45c7-a2f0-649ed3c30f1a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:13.994595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:13.995875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:13.996197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:13.996484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] PATH_INFO: `/v3/users/ab0ba257489c491ca4cd1b64d8e456a1` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:13.998532 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0ba257489c491ca4cd1b64d8e456a1)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:14.000545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:14.030618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c0e78-8f9d-4506-a44f-f3304547cdb4 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:14.034661 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08629827-6321-4d5b-976b-cab5e0b8c4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:14.035361 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629827-6321-4d5b-976b-cab5e0b8c4fc service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:14.035657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629827-6321-4d5b-976b-cab5e0b8c4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:14.035935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08629827-6321-4d5b-976b-cab5e0b8c4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:14.046175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08629827-6321-4d5b-976b-cab5e0b8c4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:14.047751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08629827-6321-4d5b-976b-cab5e0b8c4fc service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:14.058437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1326/2676] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:07:14.126740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:14.206670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0ba257489c491ca4cd1b64d8e456a1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.296557 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0ba257489c491ca4cd1b64d8e456a1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.297208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0ba257489c491ca4cd1b64d8e456a1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.304671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0ba257489c491ca4cd1b64d8e456a1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.306390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0ba257489c491ca4cd1b64d8e456a1'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.321189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:14.325821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1351/2677] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:13 2022] DELETE /identity/v3/users/ab0ba257489c491ca4cd1b64d8e456a1 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:14.335021 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70c182-2022-400e-9f7c-4da5464e85e3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:14.335309 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70c182-2022-400e-9f7c-4da5464e85e3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:14.335588 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70c182-2022-400e-9f7c-4da5464e85e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:14.355838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-ce70c182-2022-400e-9f7c-4da5464e85e3 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:14.364419 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ce70c182-2022-400e-9f7c-4da5464e85e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:14.428960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1327/2678] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:14.439014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:14.440574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:14.507014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a909d14-e2c5-40f1-b062-39fa894bb046 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:14.510973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94643934-0638-49ad-b977-9588717e5ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:14.511627 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643934-0638-49ad-b977-9588717e5ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:14.511911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643934-0638-49ad-b977-9588717e5ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:14.512221 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643934-0638-49ad-b977-9588717e5ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:14.527819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94643934-0638-49ad-b977-9588717e5ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:14.527819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94643934-0638-49ad-b977-9588717e5ac0 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:14.538000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1352/2679] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:14.657823 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70c182-2022-400e-9f7c-4da5464e85e3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:14.659755 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-ce70c182-2022-400e-9f7c-4da5464e85e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:14.688676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70c182-2022-400e-9f7c-4da5464e85e3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:14.692679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:14.693781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:14.694259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:14.694714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] PATH_INFO: `/v3/projects/a098aacb8cc147ad8b22c3dd3e9137dc` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:14.695994 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a098aacb8cc147ad8b22c3dd3e9137dc)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:14.698100 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:14.853399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a098aacb8cc147ad8b22c3dd3e9137dc', 'request_id': 'req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.932437 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a098aacb8cc147ad8b22c3dd3e9137dc', 'request_id': 'req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.945128 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a098aacb8cc147ad8b22c3dd3e9137dc', 'request_id': 'req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.957908 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] The token cache is being invalidate because project a098aacb8cc147ad8b22c3dd3e9137dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:14.958202 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.958868 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:14.960457 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1328/2680] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:14 2022] DELETE /identity/v3/projects/a098aacb8cc147ad8b22c3dd3e9137dc => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:14.964692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94643934-0638-49ad-b977-9588717e5ac0 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:14.974890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94643934-0638-49ad-b977-9588717e5ac0 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:15.011083 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:15.038752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:15.039706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:15.039999 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:15.040358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] PATH_INFO: `/v3/users/a7be5909ae564627b07ed1549251438e` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:15.041572 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7be5909ae564627b07ed1549251438e)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:15.043492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:15.101342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:15.103250 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:15.124219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cc5ec6-92d9-4c6f-ab3b-57a8f26a8068 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:15.128555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:15.129691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:15.129860 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:15.130667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] PATH_INFO: `/v3/projects/a6d19da839b9448aa7c04b0b2cdeb30f` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:15.132002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6d19da839b9448aa7c04b0b2cdeb30f)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:15.134317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:15.314574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7be5909ae564627b07ed1549251438e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.318333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6d19da839b9448aa7c04b0b2cdeb30f', 'request_id': 'req-816116ca-2059-473c-9ca0-1a7f34b189d6'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.401101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7be5909ae564627b07ed1549251438e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.401774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7be5909ae564627b07ed1549251438e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.411085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7be5909ae564627b07ed1549251438e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.411863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7be5909ae564627b07ed1549251438e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.429359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1353/2681] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:14 2022] DELETE /identity/v3/users/a7be5909ae564627b07ed1549251438e => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:15.499171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6d19da839b9448aa7c04b0b2cdeb30f', 'request_id': 'req-816116ca-2059-473c-9ca0-1a7f34b189d6'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.512188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6d19da839b9448aa7c04b0b2cdeb30f', 'request_id': 'req-816116ca-2059-473c-9ca0-1a7f34b189d6'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.523976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:15.528876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] The token cache is being invalidate because project a6d19da839b9448aa7c04b0b2cdeb30f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:15.530642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.531573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.536153 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1329/2682] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:15 2022] DELETE /identity/v3/projects/a6d19da839b9448aa7c04b0b2cdeb30f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:15.537395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:15.538840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:15.605783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b07dbe-16cc-4326-a00f-8e5e020f280c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:15.610864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:15.611847 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:15.612144 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:15.612430 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] PATH_INFO: `/v3/users/59415bfd1a8e4d6c8b846ae018af6689` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:15.613623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] RBAC: Authorizing `identity:delete_user(user_id=59415bfd1a8e4d6c8b846ae018af6689)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:15.615571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:15.788198 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59415bfd1a8e4d6c8b846ae018af6689'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.858711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:15.860324 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:15.876213 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59415bfd1a8e4d6c8b846ae018af6689'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.876951 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59415bfd1a8e4d6c8b846ae018af6689'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.878508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:15.893552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59415bfd1a8e4d6c8b846ae018af6689'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.893552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-b421adb8-13e7-48fa-bc68-0ce41a21655e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59415bfd1a8e4d6c8b846ae018af6689'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:15.915312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1354/2683] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:15 2022] DELETE /identity/v3/users/59415bfd1a8e4d6c8b846ae018af6689 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:15.936936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816116ca-2059-473c-9ca0-1a7f34b189d6 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.004952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.005859 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.006149 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.006431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] PATH_INFO: `/v3/projects/c97d1336658b4200af6817ccc1056ee3` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.007721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c97d1336658b4200af6817ccc1056ee3)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.009885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.128645 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c97d1336658b4200af6817ccc1056ee3', 'request_id': 'req-24d70049-7733-4cc1-8c9f-efe9ead2aa54'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.229253 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c97d1336658b4200af6817ccc1056ee3', 'request_id': 'req-24d70049-7733-4cc1-8c9f-efe9ead2aa54'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.236629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c97d1336658b4200af6817ccc1056ee3', 'request_id': 'req-24d70049-7733-4cc1-8c9f-efe9ead2aa54'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.248837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] The token cache is being invalidate because project c97d1336658b4200af6817ccc1056ee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:16.249164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.249882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.255439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1330/2684] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:15 2022] DELETE /identity/v3/projects/c97d1336658b4200af6817ccc1056ee3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:16.258946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e8bba-26a1-4898-b409-e6a1a38e89ca None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.262237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e8bba-26a1-4898-b409-e6a1a38e89ca None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.262237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e8bba-26a1-4898-b409-e6a1a38e89ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.278091 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-145e8bba-26a1-4898-b409-e6a1a38e89ca None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:16.279424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-145e8bba-26a1-4898-b409-e6a1a38e89ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:16.331939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.333603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:16.344085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1355/2685] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:16.352282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0347d-42bb-49b8-ab11-b3155f270a50 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.352571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0347d-42bb-49b8-ab11-b3155f270a50 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.352975 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0347d-42bb-49b8-ab11-b3155f270a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.372650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-fee0347d-42bb-49b8-ab11-b3155f270a50 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:16.374620 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-fee0347d-42bb-49b8-ab11-b3155f270a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:16.383000 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:16.390340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1356/2686] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:16.390548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0347d-42bb-49b8-ab11-b3155f270a50 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.400408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0347d-42bb-49b8-ab11-b3155f270a50 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.406886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.407571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.407857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.408151 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.408929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:16.409241 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.410567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d70049-7733-4cc1-8c9f-efe9ead2aa54 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.410967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.415225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.416134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.416432 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.416710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] PATH_INFO: `/v3/projects/e005de374bbd455cb6ec25f7c48c5f6e` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.417906 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e005de374bbd455cb6ec25f7c48c5f6e)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.420022 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.423786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1357/2687] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:07:16.428034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.438167 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf404ef-ab99-4cf2-8e84-7808bd86be0f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.443408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657910c-4138-46a7-a787-d98a9fb72e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.444063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657910c-4138-46a7-a787-d98a9fb72e1e admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.444932 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657910c-4138-46a7-a787-d98a9fb72e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.445222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657910c-4138-46a7-a787-d98a9fb72e1e admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.445977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657910c-4138-46a7-a787-d98a9fb72e1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.448375 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657910c-4138-46a7-a787-d98a9fb72e1e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.504290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e005de374bbd455cb6ec25f7c48c5f6e', 'request_id': 'req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.511043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1358/2688] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:16 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:16.515834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657910c-4138-46a7-a787-d98a9fb72e1e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.566690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e005de374bbd455cb6ec25f7c48c5f6e', 'request_id': 'req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.573037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e005de374bbd455cb6ec25f7c48c5f6e', 'request_id': 'req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.583748 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657910c-4138-46a7-a787-d98a9fb72e1e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.586403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] The token cache is being invalidate because project e005de374bbd455cb6ec25f7c48c5f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:16.587286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.587733 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.588653 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f59fb-2628-4dc5-ba76-8628a14a6273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.589328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f59fb-2628-4dc5-ba76-8628a14a6273 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.589429 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1331/2689] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:16 2022] DELETE /identity/v3/projects/e005de374bbd455cb6ec25f7c48c5f6e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:16.589611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f59fb-2628-4dc5-ba76-8628a14a6273 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.589889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f59fb-2628-4dc5-ba76-8628a14a6273 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.590170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.590670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f59fb-2628-4dc5-ba76-8628a14a6273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.592938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f59fb-2628-4dc5-ba76-8628a14a6273 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.593642 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:16.660912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a5fab-bd7d-4516-8396-e3253b5db2f2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.665608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.666486 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.666768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.667044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] PATH_INFO: `/v3/users/2d4130e9f92e41dd9118f6080098cd36` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.668279 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d4130e9f92e41dd9118f6080098cd36)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.670271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.693340 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1359/2690] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:16 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:07:16.695092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c720a-c65e-487e-aba2-fde39e2548cb None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.695379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c720a-c65e-487e-aba2-fde39e2548cb None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.696226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c720a-c65e-487e-aba2-fde39e2548cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.714619 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-d27c720a-c65e-487e-aba2-fde39e2548cb None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:16.715844 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d27c720a-c65e-487e-aba2-fde39e2548cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:16.724566 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1360/2691] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:16.725458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c720a-c65e-487e-aba2-fde39e2548cb None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.726654 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d27c720a-c65e-487e-aba2-fde39e2548cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:16.739140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c720a-c65e-487e-aba2-fde39e2548cb None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.744732 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182ddb7b-c46b-41b1-be41-57be1609be21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.745595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ddb7b-c46b-41b1-be41-57be1609be21 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.745881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ddb7b-c46b-41b1-be41-57be1609be21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.746157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182ddb7b-c46b-41b1-be41-57be1609be21 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.747181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ddb7b-c46b-41b1-be41-57be1609be21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.750701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182ddb7b-c46b-41b1-be41-57be1609be21 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.762552 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1361/2692] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:16.763275 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210f5ef-8933-44f5-ab5e-fe86153406f7 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.764216 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210f5ef-8933-44f5-ab5e-fe86153406f7 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.764494 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210f5ef-8933-44f5-ab5e-fe86153406f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.771486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:16.785373 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-1210f5ef-8933-44f5-ab5e-fe86153406f7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:16.786646 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1210f5ef-8933-44f5-ab5e-fe86153406f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:16.794974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1362/2693] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:16.796945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210f5ef-8933-44f5-ab5e-fe86153406f7 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.806815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210f5ef-8933-44f5-ab5e-fe86153406f7 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.812637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee98b92-2243-45cf-ba2d-e0b6765744ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.813561 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee98b92-2243-45cf-ba2d-e0b6765744ce admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.813977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee98b92-2243-45cf-ba2d-e0b6765744ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.814290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee98b92-2243-45cf-ba2d-e0b6765744ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.815158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee98b92-2243-45cf-ba2d-e0b6765744ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.817454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee98b92-2243-45cf-ba2d-e0b6765744ce admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.829350 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1363/2694] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:16.829997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee98b92-2243-45cf-ba2d-e0b6765744ce admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.838171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d4130e9f92e41dd9118f6080098cd36'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.840505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee98b92-2243-45cf-ba2d-e0b6765744ce admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.846308 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.847003 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.847289 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.847569 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.848341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:16.848656 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.850256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.866851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1364/2695] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:07:16.866851 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.877392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a935f3-4817-4aeb-9f78-b10078923362 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:16.883303 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504244a2-1f48-451b-a3bf-1a1f9945ae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:16.884486 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504244a2-1f48-451b-a3bf-1a1f9945ae73 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.884991 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504244a2-1f48-451b-a3bf-1a1f9945ae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.885349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504244a2-1f48-451b-a3bf-1a1f9945ae73 admin admin] PATH_INFO: `/v3/projects/41753bd464774418823ce7a74ec1c5fa/users/911f2b8037b84e3a88bb447d58227b5a/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:16.901668 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504244a2-1f48-451b-a3bf-1a1f9945ae73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41753bd464774418823ce7a74ec1c5fa, user_id=911f2b8037b84e3a88bb447d58227b5a, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:16.905049 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504244a2-1f48-451b-a3bf-1a1f9945ae73 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:16.909098 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d4130e9f92e41dd9118f6080098cd36'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.910066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d4130e9f92e41dd9118f6080098cd36'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.917455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d4130e9f92e41dd9118f6080098cd36'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.918112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d4130e9f92e41dd9118f6080098cd36'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:16.923845 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:16.935922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1332/2696] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:16 2022] DELETE /identity/v3/users/2d4130e9f92e41dd9118f6080098cd36 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:16.936825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:16.938209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:16.982067 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1365/2697] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:16 2022] PUT /identity/v3/projects/41753bd464774418823ce7a74ec1c5fa/users/911f2b8037b84e3a88bb447d58227b5a/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:16.984237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d430-e9b8-43d7-be9d-9ddba674fdd9 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:16.984535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d430-e9b8-43d7-be9d-9ddba674fdd9 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:16.984822 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4362d430-e9b8-43d7-be9d-9ddba674fdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.002610 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4362d430-e9b8-43d7-be9d-9ddba674fdd9 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:17.003284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42973269-546f-45a1-aa12-64905f178d48 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.003819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4362d430-e9b8-43d7-be9d-9ddba674fdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:17.065901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.066576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.066872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.067164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.067819 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1366/2698] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:17.067944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.069238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93335ea1-4ed7-4299-a8ba-4bd206133342 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.069530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93335ea1-4ed7-4299-a8ba-4bd206133342 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.069811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93335ea1-4ed7-4299-a8ba-4bd206133342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.069933 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.099882 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-93335ea1-4ed7-4299-a8ba-4bd206133342 None None] MFA Rules not processed for user `911f2b8037b84e3a88bb447d58227b5a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:17.101475 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-93335ea1-4ed7-4299-a8ba-4bd206133342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:17.172990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1367/2699] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:17.172990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93335ea1-4ed7-4299-a8ba-4bd206133342 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:17.174594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-93335ea1-4ed7-4299-a8ba-4bd206133342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:17.199840 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1333/2700] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:16 2022] POST /identity/v3/projects => generated 377 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:17.201640 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:17.203284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:17.221422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:17.242256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93335ea1-4ed7-4299-a8ba-4bd206133342 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.267032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171bdee3-d9a9-4f86-9ddf-cce95f33683b admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.271301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99331d-3103-4187-b944-5ac72cb79604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.271992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99331d-3103-4187-b944-5ac72cb79604 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.272330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99331d-3103-4187-b944-5ac72cb79604 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.272617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99331d-3103-4187-b944-5ac72cb79604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.298987 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40680f-ee49-4de6-a266-3196874841aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.299908 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40680f-ee49-4de6-a266-3196874841aa service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.300383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40680f-ee49-4de6-a266-3196874841aa service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.300884 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40680f-ee49-4de6-a266-3196874841aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.309334 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:17.334165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99331d-3103-4187-b944-5ac72cb79604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.335618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99331d-3103-4187-b944-5ac72cb79604 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.346774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1334/2701] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:17.353041 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99331d-3103-4187-b944-5ac72cb79604 service cinder] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:17.367888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40680f-ee49-4de6-a266-3196874841aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.369272 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40680f-ee49-4de6-a266-3196874841aa service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.371931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99331d-3103-4187-b944-5ac72cb79604 service cinder] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.378401 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff12dc-8f88-4c19-893d-83bfcd222b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.380103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff12dc-8f88-4c19-893d-83bfcd222b7a admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.380459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff12dc-8f88-4c19-893d-83bfcd222b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.380743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff12dc-8f88-4c19-893d-83bfcd222b7a admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.381174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1368/2702] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:17.389530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff12dc-8f88-4c19-893d-83bfcd222b7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.391107 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff12dc-8f88-4c19-893d-83bfcd222b7a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.498375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1335/2703] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:17 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:17.502628 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40680f-ee49-4de6-a266-3196874841aa service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:17.512413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40680f-ee49-4de6-a266-3196874841aa service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.517768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f51e50-5023-4a2a-83c2-ec903cd18342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.518516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f51e50-5023-4a2a-83c2-ec903cd18342 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.518718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f51e50-5023-4a2a-83c2-ec903cd18342 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.519000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f51e50-5023-4a2a-83c2-ec903cd18342 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.521389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f51e50-5023-4a2a-83c2-ec903cd18342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.521526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f51e50-5023-4a2a-83c2-ec903cd18342 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.533327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1369/2704] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:17.537472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff12dc-8f88-4c19-893d-83bfcd222b7a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:17.549475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff12dc-8f88-4c19-893d-83bfcd222b7a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.554207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703994f-63bf-48a2-afbb-a513542e2326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.554901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703994f-63bf-48a2-afbb-a513542e2326 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.555188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703994f-63bf-48a2-afbb-a513542e2326 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.555495 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703994f-63bf-48a2-afbb-a513542e2326 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.556260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703994f-63bf-48a2-afbb-a513542e2326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.557886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703994f-63bf-48a2-afbb-a513542e2326 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.568601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1336/2705] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:17.572606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f51e50-5023-4a2a-83c2-ec903cd18342 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:17.588973 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f51e50-5023-4a2a-83c2-ec903cd18342 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.594492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056cfce-5416-4b03-9af0-1db0eec92789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.595393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056cfce-5416-4b03-9af0-1db0eec92789 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.595679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056cfce-5416-4b03-9af0-1db0eec92789 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.595958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056cfce-5416-4b03-9af0-1db0eec92789 admin admin] PATH_INFO: `/v3/projects/859300aacc474433aa935ab4c3223a75/users/08aedc1fa28a43139ba33b6fdd77fdd6/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.612796 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056cfce-5416-4b03-9af0-1db0eec92789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=859300aacc474433aa935ab4c3223a75, user_id=08aedc1fa28a43139ba33b6fdd77fdd6, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.615512 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056cfce-5416-4b03-9af0-1db0eec92789 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.686115 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1370/2706] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:17 2022] PUT /identity/v3/projects/859300aacc474433aa935ab4c3223a75/users/08aedc1fa28a43139ba33b6fdd77fdd6/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:17.697200 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585edf1-2622-4ecb-bcfa-09842a4fd1f0 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.697497 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585edf1-2622-4ecb-bcfa-09842a4fd1f0 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.697784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585edf1-2622-4ecb-bcfa-09842a4fd1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.726781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-e585edf1-2622-4ecb-bcfa-09842a4fd1f0 None None] MFA Rules not processed for user `08aedc1fa28a43139ba33b6fdd77fdd6`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:17.728009 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e585edf1-2622-4ecb-bcfa-09842a4fd1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:17.805939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1337/2707] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:17.807811 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056cfce-5416-4b03-9af0-1db0eec92789 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:17.820376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585edf1-2622-4ecb-bcfa-09842a4fd1f0 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:17.821825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e585edf1-2622-4ecb-bcfa-09842a4fd1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:17.891353 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056cfce-5416-4b03-9af0-1db0eec92789 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.898202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e91acf-653a-4670-ac17-0d21d1894eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.898202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e91acf-653a-4670-ac17-0d21d1894eaa admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.898202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e91acf-653a-4670-ac17-0d21d1894eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.903857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e91acf-653a-4670-ac17-0d21d1894eaa admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.903857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e91acf-653a-4670-ac17-0d21d1894eaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.903857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e91acf-653a-4670-ac17-0d21d1894eaa admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.904817 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585edf1-2622-4ecb-bcfa-09842a4fd1f0 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:17.910479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:17.912751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:17.913167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:17.913543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] PATH_INFO: `/v3/projects/54ac4601e8a747698cc21d2e99b56f8f` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:17.917935 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ac4601e8a747698cc21d2e99b56f8f)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:17.919556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:17.922955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:17.999503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1371/2708] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:17 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:18.003433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e91acf-653a-4670-ac17-0d21d1894eaa admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.028658 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:18.038746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ac4601e8a747698cc21d2e99b56f8f', 'request_id': 'req-63cb8bee-2412-42ef-a46a-e20204597aba'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:18.071377 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e91acf-653a-4670-ac17-0d21d1894eaa admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.113359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ac4601e8a747698cc21d2e99b56f8f', 'request_id': 'req-63cb8bee-2412-42ef-a46a-e20204597aba'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:18.129108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ac4601e8a747698cc21d2e99b56f8f', 'request_id': 'req-63cb8bee-2412-42ef-a46a-e20204597aba'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:18.132511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.133296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.133770 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.134174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.134968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.137155 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.142443 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] The token cache is being invalidate because project 54ac4601e8a747698cc21d2e99b56f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:18.142984 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:18.143836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-63cb8bee-2412-42ef-a46a-e20204597aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:18.145782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1338/2709] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:17 2022] DELETE /identity/v3/projects/54ac4601e8a747698cc21d2e99b56f8f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:18.152344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d16b-9ebc-47c8-ac02-b3c03cf7b770 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.152637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d16b-9ebc-47c8-ac02-b3c03cf7b770 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.152918 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d16b-9ebc-47c8-ac02-b3c03cf7b770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.172377 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-9da3d16b-9ebc-47c8-ac02-b3c03cf7b770 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:18.173623 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-9da3d16b-9ebc-47c8-ac02-b3c03cf7b770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:18.182290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1339/2710] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:18.190819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58e6de-bfbe-4afe-873a-999dce6dbf3c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.191108 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58e6de-bfbe-4afe-873a-999dce6dbf3c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.191394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58e6de-bfbe-4afe-873a-999dce6dbf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.209774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-7e58e6de-bfbe-4afe-873a-999dce6dbf3c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:18.210983 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7e58e6de-bfbe-4afe-873a-999dce6dbf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:18.219547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1340/2711] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:18.223572 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58e6de-bfbe-4afe-873a-999dce6dbf3c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.232929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58e6de-bfbe-4afe-873a-999dce6dbf3c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.237570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.238218 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.238491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.238770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.239510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:18.239798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.241436 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.255161 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1341/2712] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:07:18.259668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.266481 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1372/2713] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:18 2022] POST /identity/v3/users => generated 295 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:07:18.267525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.269296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:18.271958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb5879-0f80-4838-9bde-8215f96ee0e0 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.277097 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6caec-202d-4a95-98e6-0c8fd144ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.277761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6caec-202d-4a95-98e6-0c8fd144ed30 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.278045 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6caec-202d-4a95-98e6-0c8fd144ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.278323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc6caec-202d-4a95-98e6-0c8fd144ed30 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.279403 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6caec-202d-4a95-98e6-0c8fd144ed30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.281741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc6caec-202d-4a95-98e6-0c8fd144ed30 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.281980 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d00d8b-cdb3-4d5a-9d85-b13c54f8bc29 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.287870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb159c4-34e9-42e3-9132-c2909a04e3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.288583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb159c4-34e9-42e3-9132-c2909a04e3ea admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.288855 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb159c4-34e9-42e3-9132-c2909a04e3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.289140 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb159c4-34e9-42e3-9132-c2909a04e3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.289876 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb159c4-34e9-42e3-9132-c2909a04e3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.291905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb159c4-34e9-42e3-9132-c2909a04e3ea admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.303157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1373/2714] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:18.307389 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb159c4-34e9-42e3-9132-c2909a04e3ea admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.317878 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb159c4-34e9-42e3-9132-c2909a04e3ea admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.322542 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe65a42-25c0-4835-9cd3-926fb3166391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.323857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe65a42-25c0-4835-9cd3-926fb3166391 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.323857 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe65a42-25c0-4835-9cd3-926fb3166391 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.324065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe65a42-25c0-4835-9cd3-926fb3166391 admin admin] PATH_INFO: `/v3/projects/4a5893ff03ad4963b06f69ea09c1d758/users/75d6ef85da2d49159c716288dd3b4602/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.338886 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe65a42-25c0-4835-9cd3-926fb3166391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5893ff03ad4963b06f69ea09c1d758, user_id=75d6ef85da2d49159c716288dd3b4602, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.341639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe65a42-25c0-4835-9cd3-926fb3166391 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.402870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1342/2715] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:18 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:18.406891 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6caec-202d-4a95-98e6-0c8fd144ed30 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.430059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1374/2716] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:18 2022] PUT /identity/v3/projects/4a5893ff03ad4963b06f69ea09c1d758/users/75d6ef85da2d49159c716288dd3b4602/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:18.433768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe65a42-25c0-4835-9cd3-926fb3166391 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.471328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc6caec-202d-4a95-98e6-0c8fd144ed30 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.494268 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:18.500301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe65a42-25c0-4835-9cd3-926fb3166391 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.505367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f93da3-cba1-454b-b339-a37d4d8b962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.506122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93da3-cba1-454b-b339-a37d4d8b962c admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.506406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93da3-cba1-454b-b339-a37d4d8b962c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.506883 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f93da3-cba1-454b-b339-a37d4d8b962c admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.507699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f93da3-cba1-454b-b339-a37d4d8b962c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.509376 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f93da3-cba1-454b-b339-a37d4d8b962c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.519574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1375/2717] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:18.523658 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f93da3-cba1-454b-b339-a37d4d8b962c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.533679 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f93da3-cba1-454b-b339-a37d4d8b962c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.535189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516272-2293-4155-9663-2b30016b2ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.535919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516272-2293-4155-9663-2b30016b2ebb admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.536163 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516272-2293-4155-9663-2b30016b2ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.536553 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d516272-2293-4155-9663-2b30016b2ebb admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.537251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516272-2293-4155-9663-2b30016b2ebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.541731 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a3abd-5a19-4bbf-9c4f-beeca5148ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.541866 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d516272-2293-4155-9663-2b30016b2ebb admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.541965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a3abd-5a19-4bbf-9c4f-beeca5148ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.541965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a3abd-5a19-4bbf-9c4f-beeca5148ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.541965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a3abd-5a19-4bbf-9c4f-beeca5148ec7 admin admin] PATH_INFO: `/v3/projects/4a5893ff03ad4963b06f69ea09c1d758/users/75d6ef85da2d49159c716288dd3b4602/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.542230 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a3abd-5a19-4bbf-9c4f-beeca5148ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a5893ff03ad4963b06f69ea09c1d758, user_id=75d6ef85da2d49159c716288dd3b4602, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.545148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a3abd-5a19-4bbf-9c4f-beeca5148ec7 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.555660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1376/2718] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:18 2022] PUT /identity/v3/projects/4a5893ff03ad4963b06f69ea09c1d758/users/75d6ef85da2d49159c716288dd3b4602/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:07:18.566457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5498493-13b3-4e4c-bd9c-3c65e2bf3f47 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.566750 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5498493-13b3-4e4c-bd9c-3c65e2bf3f47 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.567032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5498493-13b3-4e4c-bd9c-3c65e2bf3f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.594252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b5498493-13b3-4e4c-bd9c-3c65e2bf3f47 None None] MFA Rules not processed for user `75d6ef85da2d49159c716288dd3b4602`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:18.595477 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b5498493-13b3-4e4c-bd9c-3c65e2bf3f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:18.640260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1343/2719] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:18 2022] POST /identity/v3/users => generated 287 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:07:18.644179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516272-2293-4155-9663-2b30016b2ebb admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.656446 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d516272-2293-4155-9663-2b30016b2ebb admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.666376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:18.675626 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1377/2720] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:07:18.687401 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5498493-13b3-4e4c-bd9c-3c65e2bf3f47 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.689038 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b5498493-13b3-4e4c-bd9c-3c65e2bf3f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:18.716533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b84e9e-d084-429d-ae88-0b091dbd389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.718032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b84e9e-d084-429d-ae88-0b091dbd389e admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.718323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b84e9e-d084-429d-ae88-0b091dbd389e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.718603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b84e9e-d084-429d-ae88-0b091dbd389e admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.719327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b84e9e-d084-429d-ae88-0b091dbd389e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.720951 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b84e9e-d084-429d-ae88-0b091dbd389e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.730889 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1344/2721] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:18.735050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b84e9e-d084-429d-ae88-0b091dbd389e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.744878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b84e9e-d084-429d-ae88-0b091dbd389e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.748828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5498493-13b3-4e4c-bd9c-3c65e2bf3f47 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.750151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6324f-0631-4ef9-b051-d2fa814bfce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.750833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6324f-0631-4ef9-b051-d2fa814bfce8 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.751118 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6324f-0631-4ef9-b051-d2fa814bfce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.751438 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6324f-0631-4ef9-b051-d2fa814bfce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.752220 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6324f-0631-4ef9-b051-d2fa814bfce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.752969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0941054-fc3f-4647-b102-1462b7ca5d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.753623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0941054-fc3f-4647-b102-1462b7ca5d2b service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.753911 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0941054-fc3f-4647-b102-1462b7ca5d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.754042 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6324f-0631-4ef9-b051-d2fa814bfce8 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.754187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0941054-fc3f-4647-b102-1462b7ca5d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.756048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d0941054-fc3f-4647-b102-1462b7ca5d2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:18.769267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1345/2722] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:07:18.770283 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6324f-0631-4ef9-b051-d2fa814bfce8 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:18.781426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6324f-0631-4ef9-b051-d2fa814bfce8 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:18.786151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:18.787030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.787313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.787596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] PATH_INFO: `/v3/projects/3520876da8974c439cef1b35397029f9/users/3eca908cd8a1481c9e4c11036cd4df7b/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.804950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3520876da8974c439cef1b35397029f9, user_id=3eca908cd8a1481c9e4c11036cd4df7b, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.807624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.824455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0941054-fc3f-4647-b102-1462b7ca5d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:18.825968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0941054-fc3f-4647-b102-1462b7ca5d2b service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:18.835324 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1378/2723] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:07:18.903179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1346/2724] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:18 2022] PUT /identity/v3/projects/3520876da8974c439cef1b35397029f9/users/3eca908cd8a1481c9e4c11036cd4df7b/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:18.908612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9172-9e58-4692-9fb3-5b130ffbb3df None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:18.908976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9172-9e58-4692-9fb3-5b130ffbb3df None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:18.909327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9172-9e58-4692-9fb3-5b130ffbb3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:18.926453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ff3a9172-9e58-4692-9fb3-5b130ffbb3df None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:18.927698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ff3a9172-9e58-4692-9fb3-5b130ffbb3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:18.948381 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:19.009442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1379/2725] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:19.017096 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:19.018963 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:19.083407 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42290f3-eaaf-48e5-9c65-c419df72caf2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:19.087228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a35bd-e416-4799-8979-e64996ad5ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:19.088900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a35bd-e416-4799-8979-e64996ad5ddc service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:19.088900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a35bd-e416-4799-8979-e64996ad5ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:19.088900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a35bd-e416-4799-8979-e64996ad5ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:19.098447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a35bd-e416-4799-8979-e64996ad5ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:19.099910 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a35bd-e416-4799-8979-e64996ad5ddc service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:19.109760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1347/2726] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:24.589513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cca8b-25bb-4be8-a787-6528e751c6c5 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:24.595074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cca8b-25bb-4be8-a787-6528e751c6c5 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:24.595074 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8cca8b-25bb-4be8-a787-6528e751c6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:24.631254 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-0a8cca8b-25bb-4be8-a787-6528e751c6c5 None None] MFA Rules not processed for user `3eca908cd8a1481c9e4c11036cd4df7b`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:24.632453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0a8cca8b-25bb-4be8-a787-6528e751c6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:24.725807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1380/2727] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:24.730296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a35bd-e416-4799-8979-e64996ad5ddc service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:24.786060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a35bd-e416-4799-8979-e64996ad5ddc service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:24.791057 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf35c-fc8f-4896-ad28-f6e54a6fc9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:24.791715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf35c-fc8f-4896-ad28-f6e54a6fc9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:24.791995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf35c-fc8f-4896-ad28-f6e54a6fc9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:24.792375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1decf35c-fc8f-4896-ad28-f6e54a6fc9a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:24.793118 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decf35c-fc8f-4896-ad28-f6e54a6fc9a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:24.795898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1decf35c-fc8f-4896-ad28-f6e54a6fc9a9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:24.878838 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1348/2728] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:24 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:07:24.882950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cca8b-25bb-4be8-a787-6528e751c6c5 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:24.894979 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8cca8b-25bb-4be8-a787-6528e751c6c5 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:24.953358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c98b6d-3477-4f8f-8d22-e2a5ae40973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:24.954039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c98b6d-3477-4f8f-8d22-e2a5ae40973c admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:24.954327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c98b6d-3477-4f8f-8d22-e2a5ae40973c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:24.954606 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c98b6d-3477-4f8f-8d22-e2a5ae40973c admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:24.955306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c98b6d-3477-4f8f-8d22-e2a5ae40973c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:24.957274 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c98b6d-3477-4f8f-8d22-e2a5ae40973c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:25.030489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1381/2729] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:24 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:25.034290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf35c-fc8f-4896-ad28-f6e54a6fc9a9 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:25.041079 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:25.045066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1decf35c-fc8f-4896-ad28-f6e54a6fc9a9 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:25.049754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a02b4c-eead-43ac-88a6-a3ea8c86a9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:25.050418 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a02b4c-eead-43ac-88a6-a3ea8c86a9fb admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:25.050798 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a02b4c-eead-43ac-88a6-a3ea8c86a9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:25.051192 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a02b4c-eead-43ac-88a6-a3ea8c86a9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:25.051929 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a02b4c-eead-43ac-88a6-a3ea8c86a9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:25.053579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a02b4c-eead-43ac-88a6-a3ea8c86a9fb admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:25.064631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1349/2730] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:25.068870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c98b6d-3477-4f8f-8d22-e2a5ae40973c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:25.079547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c98b6d-3477-4f8f-8d22-e2a5ae40973c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:25.084157 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9115a-8c02-4ed8-a030-8602493023b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:25.085043 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9115a-8c02-4ed8-a030-8602493023b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:25.085440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9115a-8c02-4ed8-a030-8602493023b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:25.085724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9115a-8c02-4ed8-a030-8602493023b6 admin admin] PATH_INFO: `/v3/projects/e10757151e474c8cabd1675d8a9f9bdc/users/7493db8df3cc4195b950e6c952792cb5/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:25.095229 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:25.101870 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9115a-8c02-4ed8-a030-8602493023b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e10757151e474c8cabd1675d8a9f9bdc, user_id=7493db8df3cc4195b950e6c952792cb5, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:25.104669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9115a-8c02-4ed8-a030-8602493023b6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:25.201409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1382/2731] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:25 2022] PUT /identity/v3/projects/e10757151e474c8cabd1675d8a9f9bdc/users/7493db8df3cc4195b950e6c952792cb5/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:25.205611 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a02b4c-eead-43ac-88a6-a3ea8c86a9fb admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:25.215721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a02b4c-eead-43ac-88a6-a3ea8c86a9fb admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:25.280333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd554f6-af3e-4849-9805-eefd8813f583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:25.281036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd554f6-af3e-4849-9805-eefd8813f583 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:25.281317 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd554f6-af3e-4849-9805-eefd8813f583 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:25.281592 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd554f6-af3e-4849-9805-eefd8813f583 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:25.282368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd554f6-af3e-4849-9805-eefd8813f583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:25.283970 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd554f6-af3e-4849-9805-eefd8813f583 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:25.295186 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1350/2732] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:25.299252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9115a-8c02-4ed8-a030-8602493023b6 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:25.309900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9115a-8c02-4ed8-a030-8602493023b6 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:25.315250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:25.316413 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:25.317010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:25.317366 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] PATH_INFO: `/v3/projects/e10757151e474c8cabd1675d8a9f9bdc/users/7493db8df3cc4195b950e6c952792cb5/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:25.319609 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e10757151e474c8cabd1675d8a9f9bdc, user_id=7493db8df3cc4195b950e6c952792cb5, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:25.322503 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:25.334764 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1383/2733] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:25 2022] PUT /identity/v3/projects/e10757151e474c8cabd1675d8a9f9bdc/users/7493db8df3cc4195b950e6c952792cb5/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:31.007297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b03de-38ae-470e-8307-61181d0b035c None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:31.007865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b03de-38ae-470e-8307-61181d0b035c None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:31.007865 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b03de-38ae-470e-8307-61181d0b035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:31.037965 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-620b03de-38ae-470e-8307-61181d0b035c None None] MFA Rules not processed for user `7493db8df3cc4195b950e6c952792cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:31.038843 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-620b03de-38ae-470e-8307-61181d0b035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:31.132156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1351/2734] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:31.148217 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:31.150541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:31.281736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5653dd-8c0c-4ab6-9113-b8d0457c695c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:31.287224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91840eac-6289-42d3-9e44-dd1ac3d5bee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:31.287872 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91840eac-6289-42d3-9e44-dd1ac3d5bee2 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:31.288243 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91840eac-6289-42d3-9e44-dd1ac3d5bee2 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:31.288527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91840eac-6289-42d3-9e44-dd1ac3d5bee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:31.346849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91840eac-6289-42d3-9e44-dd1ac3d5bee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:31.348514 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91840eac-6289-42d3-9e44-dd1ac3d5bee2 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:31.358833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1384/2735] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:31.797943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b03de-38ae-470e-8307-61181d0b035c None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:31.799389 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-620b03de-38ae-470e-8307-61181d0b035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:31.871548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b03de-38ae-470e-8307-61181d0b035c None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:31.876399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f9a28-f708-4fbf-9049-ffe7172babdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:31.877214 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f9a28-f708-4fbf-9049-ffe7172babdf service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:31.879512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f9a28-f708-4fbf-9049-ffe7172babdf service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:31.879512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f9a28-f708-4fbf-9049-ffe7172babdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:31.909355 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:31.942657 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f9a28-f708-4fbf-9049-ffe7172babdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:31.944151 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f9a28-f708-4fbf-9049-ffe7172babdf service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:31.956784 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1352/2736] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:31 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:33.910577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda8ff2-3280-4c4b-93e5-4229aa8448b2 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:33.910577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda8ff2-3280-4c4b-93e5-4229aa8448b2 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:33.910577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda8ff2-3280-4c4b-93e5-4229aa8448b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:33.927924 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-6cda8ff2-3280-4c4b-93e5-4229aa8448b2 None None] MFA Rules not processed for user `86335999e9744e4c815a628607618b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:33.929250 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6cda8ff2-3280-4c4b-93e5-4229aa8448b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:33.975349 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:34.054251 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1385/2737] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:34.468467 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd9e7d-9acd-49a0-9b7a-94bac8165e8f None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:34.468955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd9e7d-9acd-49a0-9b7a-94bac8165e8f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:34.469061 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbd9e7d-9acd-49a0-9b7a-94bac8165e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:34.499295 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-3fbd9e7d-9acd-49a0-9b7a-94bac8165e8f None None] MFA Rules not processed for user `3eca908cd8a1481c9e4c11036cd4df7b`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:34.500738 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3fbd9e7d-9acd-49a0-9b7a-94bac8165e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:34.509494 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1353/2738] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:36.803304 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda8ff2-3280-4c4b-93e5-4229aa8448b2 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:36.804765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6cda8ff2-3280-4c4b-93e5-4229aa8448b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:36.896383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda8ff2-3280-4c4b-93e5-4229aa8448b2 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:36.901744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32f48a-db74-4a63-89b4-a4c17303dd34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:36.902404 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f48a-db74-4a63-89b4-a4c17303dd34 service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:36.902685 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f48a-db74-4a63-89b4-a4c17303dd34 service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:36.902960 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32f48a-db74-4a63-89b4-a4c17303dd34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:36.905767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ab32f48a-db74-4a63-89b4-a4c17303dd34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:36.917528 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32f48a-db74-4a63-89b4-a4c17303dd34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:36.919093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32f48a-db74-4a63-89b4-a4c17303dd34 service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:36.929661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1386/2739] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:44.487666 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=19 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:46.925362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=53 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:48.618169 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd9e7d-9acd-49a0-9b7a-94bac8165e8f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:48.619596 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3fbd9e7d-9acd-49a0-9b7a-94bac8165e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:48.685512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbd9e7d-9acd-49a0-9b7a-94bac8165e8f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:48.689982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:48.690855 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:48.691138 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:48.691415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] PATH_INFO: `/v3/users/911f2b8037b84e3a88bb447d58227b5a` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:48.692714 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=911f2b8037b84e3a88bb447d58227b5a)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:48.694542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:48.871626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '911f2b8037b84e3a88bb447d58227b5a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:48.949289 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '911f2b8037b84e3a88bb447d58227b5a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:48.949836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '911f2b8037b84e3a88bb447d58227b5a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:48.958760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '911f2b8037b84e3a88bb447d58227b5a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:48.961916 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '911f2b8037b84e3a88bb447d58227b5a'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:48.976701 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1354/2740] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:07:48 2022] DELETE /identity/v3/users/911f2b8037b84e3a88bb447d58227b5a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:48.981210 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf73a4-9e13-4805-ae87-1d86611fbc60 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:48.981734 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf73a4-9e13-4805-ae87-1d86611fbc60 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:48.981836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf73a4-9e13-4805-ae87-1d86611fbc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:49.009153 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-bdaf73a4-9e13-4805-ae87-1d86611fbc60 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:49.010379 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bdaf73a4-9e13-4805-ae87-1d86611fbc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:49.071901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1387/2741] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:49.083228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:49.084679 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:49.176998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85145196-1f69-424b-8878-548381eb1dd9 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:49.181598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5dfbb-468f-47c0-95c5-dc38bf481110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:49.182316 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5dfbb-468f-47c0-95c5-dc38bf481110 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:49.182603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5dfbb-468f-47c0-95c5-dc38bf481110 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:49.182884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5dfbb-468f-47c0-95c5-dc38bf481110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:49.194670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5dfbb-468f-47c0-95c5-dc38bf481110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:49.196210 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5dfbb-468f-47c0-95c5-dc38bf481110 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:49.207131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1355/2742] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:49.619204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf73a4-9e13-4805-ae87-1d86611fbc60 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:49.625969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bdaf73a4-9e13-4805-ae87-1d86611fbc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:49.634472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf73a4-9e13-4805-ae87-1d86611fbc60 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:49.639469 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:49.641964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:49.642280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:49.642562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] PATH_INFO: `/v3/projects/41753bd464774418823ce7a74ec1c5fa` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:49.643846 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41753bd464774418823ce7a74ec1c5fa)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:49.645842 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:49.773065 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41753bd464774418823ce7a74ec1c5fa', 'request_id': 'req-9ce39633-a246-45d7-8f17-d69e328aca63'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:49.875238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41753bd464774418823ce7a74ec1c5fa', 'request_id': 'req-9ce39633-a246-45d7-8f17-d69e328aca63'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:49.881196 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41753bd464774418823ce7a74ec1c5fa', 'request_id': 'req-9ce39633-a246-45d7-8f17-d69e328aca63'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:49.892881 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] The token cache is being invalidate because project 41753bd464774418823ce7a74ec1c5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:07:49.893169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:49.894369 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ce39633-a246-45d7-8f17-d69e328aca63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:07:49.895923 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1388/2743] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:07:49 2022] DELETE /identity/v3/projects/41753bd464774418823ce7a74ec1c5fa => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:49.901743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d223aa-d781-49aa-93b9-bea90f971fb8 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:49.902245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d223aa-d781-49aa-93b9-bea90f971fb8 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:49.902363 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d223aa-d781-49aa-93b9-bea90f971fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:49.905767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:49.949050 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-86d223aa-d781-49aa-93b9-bea90f971fb8 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:49.950254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-86d223aa-d781-49aa-93b9-bea90f971fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:50.010175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1356/2744] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:50.021998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5df6af-ec8c-44aa-87c0-3dd45006820c None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.022489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5df6af-ec8c-44aa-87c0-3dd45006820c None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.022624 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5df6af-ec8c-44aa-87c0-3dd45006820c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.041548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-9b5df6af-ec8c-44aa-87c0-3dd45006820c None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:50.042769 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9b5df6af-ec8c-44aa-87c0-3dd45006820c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:50.052436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1389/2745] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:50.056739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d223aa-d781-49aa-93b9-bea90f971fb8 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:50.067839 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d223aa-d781-49aa-93b9-bea90f971fb8 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:50.073256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:50.073426 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.073583 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.073863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.075173 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:07:50.075710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:50.078863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:50.090932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1357/2746] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:07:50.095390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5df6af-ec8c-44aa-87c0-3dd45006820c None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:50.104693 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5df6af-ec8c-44aa-87c0-3dd45006820c None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:50.109910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f663a-8fc5-4821-92ca-b50bb89acd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:50.110743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f663a-8fc5-4821-92ca-b50bb89acd9c admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.111031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f663a-8fc5-4821-92ca-b50bb89acd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.111310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f663a-8fc5-4821-92ca-b50bb89acd9c admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.112052 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f663a-8fc5-4821-92ca-b50bb89acd9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:50.113995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411f663a-8fc5-4821-92ca-b50bb89acd9c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:50.246867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1390/2747] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:07:50 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:50.252503 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:50.262399 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313bb7d-2eb8-40b5-acd0-fbae331cb84a admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:50.319902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511f349-ec4e-479a-ae87-45d477551429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:50.320678 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f349-ec4e-479a-ae87-45d477551429 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.320968 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f349-ec4e-479a-ae87-45d477551429 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.321274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8511f349-ec4e-479a-ae87-45d477551429 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.321990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511f349-ec4e-479a-ae87-45d477551429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:50.323980 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8511f349-ec4e-479a-ae87-45d477551429 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:50.435607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1358/2748] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:07:50 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:07:50.439782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f663a-8fc5-4821-92ca-b50bb89acd9c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:50.450130 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f663a-8fc5-4821-92ca-b50bb89acd9c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:50.455172 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee033d1-ef3d-467c-b264-8151eb61dc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:50.455828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee033d1-ef3d-467c-b264-8151eb61dc69 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.456174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee033d1-ef3d-467c-b264-8151eb61dc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.456455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee033d1-ef3d-467c-b264-8151eb61dc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.457209 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee033d1-ef3d-467c-b264-8151eb61dc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:50.458821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee033d1-ef3d-467c-b264-8151eb61dc69 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:50.470002 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1391/2749] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:50.473894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511f349-ec4e-479a-ae87-45d477551429 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:50.485134 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8511f349-ec4e-479a-ae87-45d477551429 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:50.489938 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072098a3-8e0b-45cc-93ae-025f0e93cff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:50.490613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072098a3-8e0b-45cc-93ae-025f0e93cff3 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.491143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072098a3-8e0b-45cc-93ae-025f0e93cff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.491251 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072098a3-8e0b-45cc-93ae-025f0e93cff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.491956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072098a3-8e0b-45cc-93ae-025f0e93cff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:50.493555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072098a3-8e0b-45cc-93ae-025f0e93cff3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:50.503786 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1359/2750] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:07:50.507969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee033d1-ef3d-467c-b264-8151eb61dc69 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:50.518058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee033d1-ef3d-467c-b264-8151eb61dc69 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:50.522629 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:50.523500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.523783 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.524061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] PATH_INFO: `/v3/projects/5f50cc3ec1d646a4b6ac97c0e11e8ea5/users/a34e3ccda41449de98171968bcbc2798/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.539468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f50cc3ec1d646a4b6ac97c0e11e8ea5, user_id=a34e3ccda41449de98171968bcbc2798, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:50.542122 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:50.610558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1392/2751] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:07:50 2022] PUT /identity/v3/projects/5f50cc3ec1d646a4b6ac97c0e11e8ea5/users/a34e3ccda41449de98171968bcbc2798/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:07:50.621990 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fd1f9-9848-420a-b21f-946fbe715d7a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.622282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fd1f9-9848-420a-b21f-946fbe715d7a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.622565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fd1f9-9848-420a-b21f-946fbe715d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.649116 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c03fd1f9-9848-420a-b21f-946fbe715d7a None None] MFA Rules not processed for user `a34e3ccda41449de98171968bcbc2798`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:07:50.650599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c03fd1f9-9848-420a-b21f-946fbe715d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:50.682391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:50.712917 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1360/2752] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:07:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:07:50.725555 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:50.728270 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:07:50.743962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:07:50.794176 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfa7c3-fe12-44b7-a8ae-34c14d57edce admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:07:50.797889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c0883bc26a04de2916c78df342d2023', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:07:50.799659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:07:50.799659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:07:50.799659 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:07:50.808204 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:07:50.809661 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:07:50.818549 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1393/2753] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:07:59.976477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fd1f9-9848-420a-b21f-946fbe715d7a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:07:59.983799 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c03fd1f9-9848-420a-b21f-946fbe715d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:00.075523 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fd1f9-9848-420a-b21f-946fbe715d7a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:00.081881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e183e39-d238-4f73-a79f-43ce658e554e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:00.082567 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e183e39-d238-4f73-a79f-43ce658e554e service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:00.082852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e183e39-d238-4f73-a79f-43ce658e554e service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:00.083135 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e183e39-d238-4f73-a79f-43ce658e554e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:00.087731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4e183e39-d238-4f73-a79f-43ce658e554e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:00.166740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e183e39-d238-4f73-a79f-43ce658e554e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:00.168400 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e183e39-d238-4f73-a79f-43ce658e554e service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:00.179313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1361/2754] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:07:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:00.886031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=20 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:03.117590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:03.119804 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:03.227852 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3bfb16-2433-436b-a04d-34904b59e1c1 service glance] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:03.235068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:03.236563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:03.236563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:03.238637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] PATH_INFO: `/v3/users/08aedc1fa28a43139ba33b6fdd77fdd6` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:03.240070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08aedc1fa28a43139ba33b6fdd77fdd6)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:03.242589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:03.457538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08aedc1fa28a43139ba33b6fdd77fdd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:03.566164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08aedc1fa28a43139ba33b6fdd77fdd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:03.566775 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08aedc1fa28a43139ba33b6fdd77fdd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:03.575421 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08aedc1fa28a43139ba33b6fdd77fdd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:03.576207 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08aedc1fa28a43139ba33b6fdd77fdd6'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:03.595594 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1394/2755] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:03 2022] DELETE /identity/v3/users/08aedc1fa28a43139ba33b6fdd77fdd6 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:03.600931 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2773d2-15c0-4972-8ff2-3cad932a45de None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:03.601894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2773d2-15c0-4972-8ff2-3cad932a45de None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:03.602206 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2773d2-15c0-4972-8ff2-3cad932a45de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:03.619824 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-bc2773d2-15c0-4972-8ff2-3cad932a45de None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:03.621078 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bc2773d2-15c0-4972-8ff2-3cad932a45de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:03.684248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1362/2756] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:08:03.694535 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:03.696139 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:03.755630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ebd478-e8db-4589-8370-440fcf71f768 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:03.760765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6debd-1acd-415b-99be-9f5a9a511521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:03.761461 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6debd-1acd-415b-99be-9f5a9a511521 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:03.761812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6debd-1acd-415b-99be-9f5a9a511521 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:03.762034 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6debd-1acd-415b-99be-9f5a9a511521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:03.774745 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6debd-1acd-415b-99be-9f5a9a511521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:03.775131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6debd-1acd-415b-99be-9f5a9a511521 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:03.786037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1395/2757] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:04.238198 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2773d2-15c0-4972-8ff2-3cad932a45de None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:04.243222 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bc2773d2-15c0-4972-8ff2-3cad932a45de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:04.256196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2773d2-15c0-4972-8ff2-3cad932a45de None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:04.263358 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:04.264922 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:04.265290 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:04.265956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] PATH_INFO: `/v3/projects/859300aacc474433aa935ab4c3223a75` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:04.274558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=859300aacc474433aa935ab4c3223a75)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:04.274558 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:04.301864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=100 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:04.421121 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '859300aacc474433aa935ab4c3223a75', 'request_id': 'req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.493356 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '859300aacc474433aa935ab4c3223a75', 'request_id': 'req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.499745 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '859300aacc474433aa935ab4c3223a75', 'request_id': 'req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.511427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] The token cache is being invalidate because project 859300aacc474433aa935ab4c3223a75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:04.511723 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.512474 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.514165 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1363/2758] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:04 2022] DELETE /identity/v3/projects/859300aacc474433aa935ab4c3223a75 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:04.518227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6debd-1acd-415b-99be-9f5a9a511521 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:04.529005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6debd-1acd-415b-99be-9f5a9a511521 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:04.548735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:04.589183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:04.590078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:04.590359 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:04.590639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] PATH_INFO: `/v3/users/75d6ef85da2d49159c716288dd3b4602` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:04.592022 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75d6ef85da2d49159c716288dd3b4602)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:04.594023 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:04.778056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75d6ef85da2d49159c716288dd3b4602'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.839240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75d6ef85da2d49159c716288dd3b4602'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.840281 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75d6ef85da2d49159c716288dd3b4602'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.848915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75d6ef85da2d49159c716288dd3b4602'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.849690 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ea873ab4-ddf4-418c-9666-2282076395d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75d6ef85da2d49159c716288dd3b4602'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:04.875241 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1396/2759] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:04 2022] DELETE /identity/v3/users/75d6ef85da2d49159c716288dd3b4602 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:05.332602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:05.332602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:05.348260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc67d-dd30-4345-bba5-5dfebecc5ebe admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:05.420424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:05.421305 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:05.421591 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:05.421864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] PATH_INFO: `/v3/projects/4a5893ff03ad4963b06f69ea09c1d758` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:05.423174 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a5893ff03ad4963b06f69ea09c1d758)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:05.425132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:05.535793 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a5893ff03ad4963b06f69ea09c1d758', 'request_id': 'req-73330d48-89a1-4647-962b-d4722d248ecc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:05.595180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a5893ff03ad4963b06f69ea09c1d758', 'request_id': 'req-73330d48-89a1-4647-962b-d4722d248ecc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:05.602273 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a5893ff03ad4963b06f69ea09c1d758', 'request_id': 'req-73330d48-89a1-4647-962b-d4722d248ecc'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:05.614652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] The token cache is being invalidate because project 4a5893ff03ad4963b06f69ea09c1d758 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:05.614998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:05.616167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-73330d48-89a1-4647-962b-d4722d248ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:05.618518 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1364/2760] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:05 2022] DELETE /identity/v3/projects/4a5893ff03ad4963b06f69ea09c1d758 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:05.623326 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10a1c-ac2e-4d6a-8561-9a2d734e5b5e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:05.623840 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10a1c-ac2e-4d6a-8561-9a2d734e5b5e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:05.623955 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e10a1c-ac2e-4d6a-8561-9a2d734e5b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:05.643786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-74e10a1c-ac2e-4d6a-8561-9a2d734e5b5e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:05.644848 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-74e10a1c-ac2e-4d6a-8561-9a2d734e5b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:05.709513 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1397/2761] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:05.717546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0566-5194-4693-9a2d-db14ac930889 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:05.717901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0566-5194-4693-9a2d-db14ac930889 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:05.718278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f0566-5194-4693-9a2d-db14ac930889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:05.736631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-6e7f0566-5194-4693-9a2d-db14ac930889 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:05.737833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-6e7f0566-5194-4693-9a2d-db14ac930889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:05.746533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1365/2762] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:05.753231 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e10a1c-ac2e-4d6a-8561-9a2d734e5b5e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:05.765969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e10a1c-ac2e-4d6a-8561-9a2d734e5b5e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:05.771536 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:05.772244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:05.773102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:05.773102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:05.773699 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:08:05.774089 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:05.775996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:05.788266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1398/2763] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:08:05.792420 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f0566-5194-4693-9a2d-db14ac930889 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:05.801735 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f0566-5194-4693-9a2d-db14ac930889 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:05.807033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abf09a-0bbc-46de-843f-5f2a5a4e90d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:05.807692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abf09a-0bbc-46de-843f-5f2a5a4e90d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:05.807976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abf09a-0bbc-46de-843f-5f2a5a4e90d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:05.808307 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47abf09a-0bbc-46de-843f-5f2a5a4e90d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:05.809013 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abf09a-0bbc-46de-843f-5f2a5a4e90d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:05.810919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47abf09a-0bbc-46de-843f-5f2a5a4e90d8 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:05.921184 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1366/2764] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:05 2022] POST /identity/v3/projects => generated 401 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:05.925403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:05.936488 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea331-48c6-400b-9e5d-22f61dba5f65 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:05.961696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:05.992996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f91e17-33cf-4c47-83e4-84dce020a206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:05.993682 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f91e17-33cf-4c47-83e4-84dce020a206 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:05.993967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f91e17-33cf-4c47-83e4-84dce020a206 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:05.994246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f91e17-33cf-4c47-83e4-84dce020a206 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:05.994948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f91e17-33cf-4c47-83e4-84dce020a206 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:05.997076 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f91e17-33cf-4c47-83e4-84dce020a206 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:06.051652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1399/2765] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:05 2022] POST /identity/v3/users => generated 301 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:06.061756 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abf09a-0bbc-46de-843f-5f2a5a4e90d8 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:06.072978 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47abf09a-0bbc-46de-843f-5f2a5a4e90d8 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:06.077619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358cac-dbfc-40f2-9fdf-368dc11256ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:06.078327 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358cac-dbfc-40f2-9fdf-368dc11256ba admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:06.078608 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358cac-dbfc-40f2-9fdf-368dc11256ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:06.078888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99358cac-dbfc-40f2-9fdf-368dc11256ba admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:06.079668 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99358cac-dbfc-40f2-9fdf-368dc11256ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:06.081373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99358cac-dbfc-40f2-9fdf-368dc11256ba admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:06.083674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:06.093711 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1367/2766] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:06.097538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f91e17-33cf-4c47-83e4-84dce020a206 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:06.108385 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f91e17-33cf-4c47-83e4-84dce020a206 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:06.113518 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc41c8c-7f7f-4e4c-9f26-139e938b8d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:06.114306 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc41c8c-7f7f-4e4c-9f26-139e938b8d64 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:06.114595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc41c8c-7f7f-4e4c-9f26-139e938b8d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:06.114873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc41c8c-7f7f-4e4c-9f26-139e938b8d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:06.115612 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc41c8c-7f7f-4e4c-9f26-139e938b8d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:06.118097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc41c8c-7f7f-4e4c-9f26-139e938b8d64 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:06.128454 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1400/2767] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:06.132570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358cac-dbfc-40f2-9fdf-368dc11256ba admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:06.142425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99358cac-dbfc-40f2-9fdf-368dc11256ba admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:06.147168 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa091eb-6625-40a6-b4b1-1833b4d03d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:06.148272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa091eb-6625-40a6-b4b1-1833b4d03d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:06.148562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa091eb-6625-40a6-b4b1-1833b4d03d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:06.148864 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa091eb-6625-40a6-b4b1-1833b4d03d44 admin admin] PATH_INFO: `/v3/projects/3066beae7507462bb25a9bd3953a1e22/users/d2410236963b4b2d9af9cf8dd63a84c7/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:06.164319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa091eb-6625-40a6-b4b1-1833b4d03d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3066beae7507462bb25a9bd3953a1e22, user_id=d2410236963b4b2d9af9cf8dd63a84c7, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:06.167140 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa091eb-6625-40a6-b4b1-1833b4d03d44 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:06.233694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1368/2768] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:06 2022] PUT /identity/v3/projects/3066beae7507462bb25a9bd3953a1e22/users/d2410236963b4b2d9af9cf8dd63a84c7/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:08:06.245208 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92cc0f-3d0e-4170-9573-05474646cff6 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:06.245529 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92cc0f-3d0e-4170-9573-05474646cff6 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:06.245810 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92cc0f-3d0e-4170-9573-05474646cff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:06.271815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-af92cc0f-3d0e-4170-9573-05474646cff6 None None] MFA Rules not processed for user `d2410236963b4b2d9af9cf8dd63a84c7`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:06.273669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-af92cc0f-3d0e-4170-9573-05474646cff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:06.337591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1401/2769] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:06.342020 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa091eb-6625-40a6-b4b1-1833b4d03d44 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:06.403667 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa091eb-6625-40a6-b4b1-1833b4d03d44 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:06.408493 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ff50d-fdf3-4236-ad4e-66aaf095398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:06.409188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ff50d-fdf3-4236-ad4e-66aaf095398c admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:06.409604 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ff50d-fdf3-4236-ad4e-66aaf095398c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:06.409899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ff50d-fdf3-4236-ad4e-66aaf095398c admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:06.410602 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ff50d-fdf3-4236-ad4e-66aaf095398c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:06.412576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307ff50d-fdf3-4236-ad4e-66aaf095398c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:06.522439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1369/2770] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:06 2022] POST /identity/v3/projects => generated 401 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:06.527338 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92cc0f-3d0e-4170-9573-05474646cff6 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:06.538056 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92cc0f-3d0e-4170-9573-05474646cff6 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:06.597120 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442ee39-af95-4532-9a11-b5ca0bceb528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:06.597728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442ee39-af95-4532-9a11-b5ca0bceb528 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:06.598009 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442ee39-af95-4532-9a11-b5ca0bceb528 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:06.598287 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442ee39-af95-4532-9a11-b5ca0bceb528 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:06.598985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442ee39-af95-4532-9a11-b5ca0bceb528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:06.601017 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442ee39-af95-4532-9a11-b5ca0bceb528 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:06.727380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1402/2771] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:06 2022] POST /identity/v3/users => generated 307 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:06.731313 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ff50d-fdf3-4236-ad4e-66aaf095398c admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:06.741834 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ff50d-fdf3-4236-ad4e-66aaf095398c admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:06.746599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86209f4-e7c9-4e57-9a99-58b466a24440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:06.747264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86209f4-e7c9-4e57-9a99-58b466a24440 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:06.747548 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86209f4-e7c9-4e57-9a99-58b466a24440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:06.747825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86209f4-e7c9-4e57-9a99-58b466a24440 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:06.748589 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86209f4-e7c9-4e57-9a99-58b466a24440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:06.750187 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86209f4-e7c9-4e57-9a99-58b466a24440 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:06.764912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1370/2772] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:06.771577 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442ee39-af95-4532-9a11-b5ca0bceb528 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:06.783177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442ee39-af95-4532-9a11-b5ca0bceb528 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:06.788030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0c8ec-43da-461f-b0c4-cf8533eaa48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:06.789258 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0c8ec-43da-461f-b0c4-cf8533eaa48e admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:06.789548 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0c8ec-43da-461f-b0c4-cf8533eaa48e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:06.789827 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0c8ec-43da-461f-b0c4-cf8533eaa48e admin admin] PATH_INFO: `/v3/projects/589f891cf6dd406ebd54874f167d9aa4/users/3009dad502a7407b80c586afa802a29f/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:06.926785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0c8ec-43da-461f-b0c4-cf8533eaa48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=589f891cf6dd406ebd54874f167d9aa4, user_id=3009dad502a7407b80c586afa802a29f, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:06.929989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b0c8ec-43da-461f-b0c4-cf8533eaa48e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:06.993834 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1403/2773] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:06 2022] PUT /identity/v3/projects/589f891cf6dd406ebd54874f167d9aa4/users/3009dad502a7407b80c586afa802a29f/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:06.997888 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86209f4-e7c9-4e57-9a99-58b466a24440 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:07.008820 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86209f4-e7c9-4e57-9a99-58b466a24440 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:07.064359 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e537a-1418-4c7b-a8c3-056d87f2e5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:07.065030 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e537a-1418-4c7b-a8c3-056d87f2e5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:07.065318 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e537a-1418-4c7b-a8c3-056d87f2e5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:07.065600 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e537a-1418-4c7b-a8c3-056d87f2e5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:07.066348 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e537a-1418-4c7b-a8c3-056d87f2e5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:07.067925 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e537a-1418-4c7b-a8c3-056d87f2e5a9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:07.079342 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1371/2774] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:07.083598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0c8ec-43da-461f-b0c4-cf8533eaa48e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:07.089319 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:07.094558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0c8ec-43da-461f-b0c4-cf8533eaa48e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:07.099202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:07.100070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:07.100387 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:07.100706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] PATH_INFO: `/v3/projects/589f891cf6dd406ebd54874f167d9aa4/users/3009dad502a7407b80c586afa802a29f/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:07.102800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=589f891cf6dd406ebd54874f167d9aa4, user_id=3009dad502a7407b80c586afa802a29f, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:07.105441 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:07.115639 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1404/2775] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:07 2022] PUT /identity/v3/projects/589f891cf6dd406ebd54874f167d9aa4/users/3009dad502a7407b80c586afa802a29f/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:07.126506 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29789d03-5d5d-4ca8-b52c-1b5109a6acf3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:07.126876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29789d03-5d5d-4ca8-b52c-1b5109a6acf3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:07.127245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29789d03-5d5d-4ca8-b52c-1b5109a6acf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:07.156256 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-29789d03-5d5d-4ca8-b52c-1b5109a6acf3 None None] MFA Rules not processed for user `3009dad502a7407b80c586afa802a29f`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:07.157483 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-29789d03-5d5d-4ca8-b52c-1b5109a6acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:07.190731 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:07.225743 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1372/2776] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:07 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:07.236867 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:07.238608 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:07.316962 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff571162-bb5d-462d-94b8-4b2d3318b259 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:07.321602 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35079796-7838-400c-859b-59bb84e60bd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:07.322269 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079796-7838-400c-859b-59bb84e60bd2 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:07.322537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079796-7838-400c-859b-59bb84e60bd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:07.322841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079796-7838-400c-859b-59bb84e60bd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:07.332085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35079796-7838-400c-859b-59bb84e60bd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:07.333547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35079796-7838-400c-859b-59bb84e60bd2 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:07.344587 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1405/2777] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:07 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:07.495863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29789d03-5d5d-4ca8-b52c-1b5109a6acf3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:07.497530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-29789d03-5d5d-4ca8-b52c-1b5109a6acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:07.510036 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29789d03-5d5d-4ca8-b52c-1b5109a6acf3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:07.513677 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729f684-3869-4576-b36d-5e57026e862b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:07.514444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f684-3869-4576-b36d-5e57026e862b service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:07.514736 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f684-3869-4576-b36d-5e57026e862b service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:07.515011 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729f684-3869-4576-b36d-5e57026e862b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:07.579901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729f684-3869-4576-b36d-5e57026e862b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:07.579901 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729f684-3869-4576-b36d-5e57026e862b service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:07.584761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1373/2778] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:15.613815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35079796-7838-400c-859b-59bb84e60bd2 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:15.685849 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35079796-7838-400c-859b-59bb84e60bd2 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:15.690630 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:15.691660 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:15.692042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:15.692374 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] PATH_INFO: `/v3/users/d2410236963b4b2d9af9cf8dd63a84c7` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:15.694146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2410236963b4b2d9af9cf8dd63a84c7)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:15.696958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:15.906192 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2410236963b4b2d9af9cf8dd63a84c7'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:15.976264 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2410236963b4b2d9af9cf8dd63a84c7'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:15.977573 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2410236963b4b2d9af9cf8dd63a84c7'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:15.985963 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2410236963b4b2d9af9cf8dd63a84c7'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:15.986607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2410236963b4b2d9af9cf8dd63a84c7'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:15.994825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:16.004473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1406/2779] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:15 2022] DELETE /identity/v3/users/d2410236963b4b2d9af9cf8dd63a84c7 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:16.009016 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36c1e9-4d63-45ee-9056-8f7301002ad3 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:16.009511 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36c1e9-4d63-45ee-9056-8f7301002ad3 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:16.009612 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36c1e9-4d63-45ee-9056-8f7301002ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:16.028851 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-fa36c1e9-4d63-45ee-9056-8f7301002ad3 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:16.030072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fa36c1e9-4d63-45ee-9056-8f7301002ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:16.096741 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1374/2780] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:16.107500 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:16.111314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:16.167168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b624d-9462-4c26-a6fd-3dffd4a9dd1e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:16.170826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08cad9-b315-4cf4-b250-edc52396ffa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:16.171474 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08cad9-b315-4cf4-b250-edc52396ffa8 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:16.171766 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08cad9-b315-4cf4-b250-edc52396ffa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:16.172044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08cad9-b315-4cf4-b250-edc52396ffa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:16.181631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08cad9-b315-4cf4-b250-edc52396ffa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:16.183104 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08cad9-b315-4cf4-b250-edc52396ffa8 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:16.193039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1407/2781] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:16.595770 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36c1e9-4d63-45ee-9056-8f7301002ad3 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:16.597673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fa36c1e9-4d63-45ee-9056-8f7301002ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:16.611270 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36c1e9-4d63-45ee-9056-8f7301002ad3 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:16.617323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:16.619341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:16.619595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:16.619875 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] PATH_INFO: `/v3/projects/3066beae7507462bb25a9bd3953a1e22` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:16.621759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3066beae7507462bb25a9bd3953a1e22)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:16.626180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:16.736559 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3066beae7507462bb25a9bd3953a1e22', 'request_id': 'req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:16.838958 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3066beae7507462bb25a9bd3953a1e22', 'request_id': 'req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:16.844147 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:16.845352 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3066beae7507462bb25a9bd3953a1e22', 'request_id': 'req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:16.856637 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] The token cache is being invalidate because project 3066beae7507462bb25a9bd3953a1e22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:16.856928 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:16.857556 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:16.859144 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1375/2782] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:16 2022] DELETE /identity/v3/projects/3066beae7507462bb25a9bd3953a1e22 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:16.863227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08cad9-b315-4cf4-b250-edc52396ffa8 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:16.873492 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08cad9-b315-4cf4-b250-edc52396ffa8 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:16.929131 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:16.930563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:16.931059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:16.931341 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] PATH_INFO: `/v3/users/3009dad502a7407b80c586afa802a29f` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:16.932603 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3009dad502a7407b80c586afa802a29f)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:16.934504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:17.109772 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3009dad502a7407b80c586afa802a29f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.189132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3009dad502a7407b80c586afa802a29f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.189746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3009dad502a7407b80c586afa802a29f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.197841 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3009dad502a7407b80c586afa802a29f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.200938 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-20656e11-c46c-4bfc-9230-3273e4e8dc80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3009dad502a7407b80c586afa802a29f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.216742 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1408/2783] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:16 2022] DELETE /identity/v3/users/3009dad502a7407b80c586afa802a29f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:17.613676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:17.615278 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:17.631007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d5d08-4312-4cb3-98c7-17bfc587c6f9 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:17.691447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:17.692345 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:17.692630 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:17.692905 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] PATH_INFO: `/v3/projects/589f891cf6dd406ebd54874f167d9aa4` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:17.694082 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=589f891cf6dd406ebd54874f167d9aa4)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:17.696104 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:17.796133 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589f891cf6dd406ebd54874f167d9aa4', 'request_id': 'req-566fd058-e467-4598-912d-a7906a746bb7'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.869292 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589f891cf6dd406ebd54874f167d9aa4', 'request_id': 'req-566fd058-e467-4598-912d-a7906a746bb7'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.874953 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589f891cf6dd406ebd54874f167d9aa4', 'request_id': 'req-566fd058-e467-4598-912d-a7906a746bb7'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.886482 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] The token cache is being invalidate because project 589f891cf6dd406ebd54874f167d9aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:17.886768 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.887402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-566fd058-e467-4598-912d-a7906a746bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:17.888992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1376/2784] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:17 2022] DELETE /identity/v3/projects/589f891cf6dd406ebd54874f167d9aa4 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:08:17.894195 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304b74-defa-4f97-bbe5-558bfac0900e None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:17.894670 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304b74-defa-4f97-bbe5-558bfac0900e None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:17.894787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13304b74-defa-4f97-bbe5-558bfac0900e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:17.913824 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-13304b74-defa-4f97-bbe5-558bfac0900e None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:17.915036 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-13304b74-defa-4f97-bbe5-558bfac0900e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:17.923333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:17.973463 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1409/2785] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:17.982543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d1bf8-1261-471b-aac4-ac9ea61963aa None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:17.982835 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d1bf8-1261-471b-aac4-ac9ea61963aa None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:17.983119 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d1bf8-1261-471b-aac4-ac9ea61963aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.001301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-bb8d1bf8-1261-471b-aac4-ac9ea61963aa None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:18.002510 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bb8d1bf8-1261-471b-aac4-ac9ea61963aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:18.011276 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1377/2786] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:18.015394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13304b74-defa-4f97-bbe5-558bfac0900e None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.025681 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13304b74-defa-4f97-bbe5-558bfac0900e None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.030181 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.030832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.031116 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.031390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.032169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:08:18.032757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.034526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.046947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1410/2787] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:08:18.051204 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d1bf8-1261-471b-aac4-ac9ea61963aa None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.060902 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d1bf8-1261-471b-aac4-ac9ea61963aa None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.065557 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573f74-456d-40b5-852c-1909361e9dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.066208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573f74-456d-40b5-852c-1909361e9dbc admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.066491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573f74-456d-40b5-852c-1909361e9dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.066769 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573f74-456d-40b5-852c-1909361e9dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.067470 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573f74-456d-40b5-852c-1909361e9dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.069397 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f573f74-456d-40b5-852c-1909361e9dbc admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.156857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1378/2788] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:18 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:18.161070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.171631 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0310d-3b87-47fd-a9a8-afaa7f3e667d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.229683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde5b8e-0a7b-4905-b9bf-b3626adfeb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.230354 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde5b8e-0a7b-4905-b9bf-b3626adfeb1b admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.230640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde5b8e-0a7b-4905-b9bf-b3626adfeb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.230916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde5b8e-0a7b-4905-b9bf-b3626adfeb1b admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.231600 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde5b8e-0a7b-4905-b9bf-b3626adfeb1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.233617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde5b8e-0a7b-4905-b9bf-b3626adfeb1b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.343730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1411/2789] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:18 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:08:18.347962 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573f74-456d-40b5-852c-1909361e9dbc admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.359066 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573f74-456d-40b5-852c-1909361e9dbc admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.363598 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59263ad3-b874-42bd-8337-f88f0d92c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.364281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59263ad3-b874-42bd-8337-f88f0d92c1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.364566 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59263ad3-b874-42bd-8337-f88f0d92c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.364885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59263ad3-b874-42bd-8337-f88f0d92c1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.365599 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59263ad3-b874-42bd-8337-f88f0d92c1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.367182 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59263ad3-b874-42bd-8337-f88f0d92c1d7 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.378414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1379/2790] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:18.382290 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde5b8e-0a7b-4905-b9bf-b3626adfeb1b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.393468 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde5b8e-0a7b-4905-b9bf-b3626adfeb1b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.398291 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34583324-66b5-4b9f-bb03-444f428fdbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.399025 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34583324-66b5-4b9f-bb03-444f428fdbd6 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.399703 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34583324-66b5-4b9f-bb03-444f428fdbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.399984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34583324-66b5-4b9f-bb03-444f428fdbd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.400743 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34583324-66b5-4b9f-bb03-444f428fdbd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.402327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34583324-66b5-4b9f-bb03-444f428fdbd6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.412419 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1412/2791] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:18.417710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59263ad3-b874-42bd-8337-f88f0d92c1d7 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.427636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59263ad3-b874-42bd-8337-f88f0d92c1d7 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.433463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5ca63-5759-4208-9e89-5b41204f0a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.434379 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5ca63-5759-4208-9e89-5b41204f0a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.434691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5ca63-5759-4208-9e89-5b41204f0a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.434971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5ca63-5759-4208-9e89-5b41204f0a9e admin admin] PATH_INFO: `/v3/projects/c141d6bc5e7e47e7b5b93ae029597651/users/942860bb17574aee848a0b3c9d22d2c9/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.450967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5ca63-5759-4208-9e89-5b41204f0a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c141d6bc5e7e47e7b5b93ae029597651, user_id=942860bb17574aee848a0b3c9d22d2c9, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.453685 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5ca63-5759-4208-9e89-5b41204f0a9e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.521298 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1380/2792] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:18 2022] PUT /identity/v3/projects/c141d6bc5e7e47e7b5b93ae029597651/users/942860bb17574aee848a0b3c9d22d2c9/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:08:18.532348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202d35-3fa7-4980-8794-0901eb02f634 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.532641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202d35-3fa7-4980-8794-0901eb02f634 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.532919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202d35-3fa7-4980-8794-0901eb02f634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.560476 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-b9202d35-3fa7-4980-8794-0901eb02f634 None None] MFA Rules not processed for user `942860bb17574aee848a0b3c9d22d2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:18.561683 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b9202d35-3fa7-4980-8794-0901eb02f634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:18.624900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1413/2793] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:18.629310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5ca63-5759-4208-9e89-5b41204f0a9e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.650383 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:18.693529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5ca63-5759-4208-9e89-5b41204f0a9e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.699857 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aa21d-17f8-457c-a62b-a7b38c0141ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.700555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aa21d-17f8-457c-a62b-a7b38c0141ba admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.700837 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aa21d-17f8-457c-a62b-a7b38c0141ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.701111 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60aa21d-17f8-457c-a62b-a7b38c0141ba admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.701808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60aa21d-17f8-457c-a62b-a7b38c0141ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.703718 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60aa21d-17f8-457c-a62b-a7b38c0141ba admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.767215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1381/2794] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:18 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:18.771363 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202d35-3fa7-4980-8794-0901eb02f634 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.783422 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9202d35-3fa7-4980-8794-0901eb02f634 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.788030 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:18.840976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660db1b1-bc02-4e2c-9df5-e14056e1bff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.841650 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660db1b1-bc02-4e2c-9df5-e14056e1bff6 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.841937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660db1b1-bc02-4e2c-9df5-e14056e1bff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.842219 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660db1b1-bc02-4e2c-9df5-e14056e1bff6 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.842919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660db1b1-bc02-4e2c-9df5-e14056e1bff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.844888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660db1b1-bc02-4e2c-9df5-e14056e1bff6 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.908522 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1414/2795] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:18 2022] POST /identity/v3/users => generated 297 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:18.912652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aa21d-17f8-457c-a62b-a7b38c0141ba admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.923442 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60aa21d-17f8-457c-a62b-a7b38c0141ba admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.928375 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898fd27a-a446-474b-a6a4-d59a30058a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.929049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fd27a-a446-474b-a6a4-d59a30058a68 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.929337 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fd27a-a446-474b-a6a4-d59a30058a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.929619 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fd27a-a446-474b-a6a4-d59a30058a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.930897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898fd27a-a446-474b-a6a4-d59a30058a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.932529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898fd27a-a446-474b-a6a4-d59a30058a68 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:18.944323 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1382/2796] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:18.949092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660db1b1-bc02-4e2c-9df5-e14056e1bff6 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:18.958985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660db1b1-bc02-4e2c-9df5-e14056e1bff6 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:18.963707 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e068949-7d47-4fa6-8bd2-8bceb1829934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:18.964611 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068949-7d47-4fa6-8bd2-8bceb1829934 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:18.964895 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068949-7d47-4fa6-8bd2-8bceb1829934 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:18.965168 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e068949-7d47-4fa6-8bd2-8bceb1829934 admin admin] PATH_INFO: `/v3/projects/6821b2b15b7741fe84a66ca5a133b51d/users/9bf683bf9ea545928e7fcf2cf825dec0/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:18.979758 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e068949-7d47-4fa6-8bd2-8bceb1829934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6821b2b15b7741fe84a66ca5a133b51d, user_id=9bf683bf9ea545928e7fcf2cf825dec0, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:18.982429 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e068949-7d47-4fa6-8bd2-8bceb1829934 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:19.052406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1415/2797] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:18 2022] PUT /identity/v3/projects/6821b2b15b7741fe84a66ca5a133b51d/users/9bf683bf9ea545928e7fcf2cf825dec0/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:19.056886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898fd27a-a446-474b-a6a4-d59a30058a68 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:19.066674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898fd27a-a446-474b-a6a4-d59a30058a68 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:19.121792 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d393e69a-5f61-4a4c-a48d-bcd60a94d1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:19.122459 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393e69a-5f61-4a4c-a48d-bcd60a94d1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:19.122739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393e69a-5f61-4a4c-a48d-bcd60a94d1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:19.123040 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d393e69a-5f61-4a4c-a48d-bcd60a94d1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:19.123754 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393e69a-5f61-4a4c-a48d-bcd60a94d1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:19.125416 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d393e69a-5f61-4a4c-a48d-bcd60a94d1d4 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:19.135721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1383/2798] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:19.139899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e068949-7d47-4fa6-8bd2-8bceb1829934 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:19.150442 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e068949-7d47-4fa6-8bd2-8bceb1829934 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:19.155039 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:19.155903 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:19.156222 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:19.156493 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] PATH_INFO: `/v3/projects/6821b2b15b7741fe84a66ca5a133b51d/users/9bf683bf9ea545928e7fcf2cf825dec0/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:19.158465 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6821b2b15b7741fe84a66ca5a133b51d, user_id=9bf683bf9ea545928e7fcf2cf825dec0, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:19.161183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:19.171152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1416/2799] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:19 2022] PUT /identity/v3/projects/6821b2b15b7741fe84a66ca5a133b51d/users/9bf683bf9ea545928e7fcf2cf825dec0/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:19.184287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490c124-a419-4cf3-8cc1-bb1e65a2d91a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:19.184287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490c124-a419-4cf3-8cc1-bb1e65a2d91a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:19.184287 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c490c124-a419-4cf3-8cc1-bb1e65a2d91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:19.209037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-c490c124-a419-4cf3-8cc1-bb1e65a2d91a None None] MFA Rules not processed for user `9bf683bf9ea545928e7fcf2cf825dec0`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:19.210231 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c490c124-a419-4cf3-8cc1-bb1e65a2d91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:19.277145 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1384/2800] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:19.289007 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:19.291249 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:19.348409 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe9ba2d-1417-426d-a234-6e88df607b9b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:19.352641 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f7780-ec40-4de1-966b-fa3e52f9745c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:19.353282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f7780-ec40-4de1-966b-fa3e52f9745c service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:19.353562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f7780-ec40-4de1-966b-fa3e52f9745c service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:19.353833 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f7780-ec40-4de1-966b-fa3e52f9745c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:19.362786 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f7780-ec40-4de1-966b-fa3e52f9745c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:19.364242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f7780-ec40-4de1-966b-fa3e52f9745c service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:19.375266 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1417/2801] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:19.509037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490c124-a419-4cf3-8cc1-bb1e65a2d91a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:19.518709 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c490c124-a419-4cf3-8cc1-bb1e65a2d91a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:19.522261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bce196-72b6-49fe-8d86-0ba75f16c576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:19.522895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce196-72b6-49fe-8d86-0ba75f16c576 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:19.523178 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce196-72b6-49fe-8d86-0ba75f16c576 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:19.523456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce196-72b6-49fe-8d86-0ba75f16c576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:19.533132 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:19.581562 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bce196-72b6-49fe-8d86-0ba75f16c576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:19.583017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bce196-72b6-49fe-8d86-0ba75f16c576 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:19.595183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1385/2802] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:20.030093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f7780-ec40-4de1-966b-fa3e52f9745c service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:20.093726 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f7780-ec40-4de1-966b-fa3e52f9745c service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:20.098464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:20.102391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:20.102391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:20.102391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] PATH_INFO: `/v3/users/942860bb17574aee848a0b3c9d22d2c9` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:20.102391 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=942860bb17574aee848a0b3c9d22d2c9)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:20.103874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:20.108657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:20.329328 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942860bb17574aee848a0b3c9d22d2c9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:20.389305 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942860bb17574aee848a0b3c9d22d2c9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:20.389918 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942860bb17574aee848a0b3c9d22d2c9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:20.397416 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942860bb17574aee848a0b3c9d22d2c9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:20.398042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942860bb17574aee848a0b3c9d22d2c9'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:20.417747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1418/2803] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:20 2022] DELETE /identity/v3/users/942860bb17574aee848a0b3c9d22d2c9 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:20.422237 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0e18f-e22d-4b9c-9bb8-3d7063a6cd41 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:20.422729 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0e18f-e22d-4b9c-9bb8-3d7063a6cd41 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:20.422854 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0e18f-e22d-4b9c-9bb8-3d7063a6cd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:20.441773 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d3f0e18f-e22d-4b9c-9bb8-3d7063a6cd41 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:20.442746 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d3f0e18f-e22d-4b9c-9bb8-3d7063a6cd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:20.501297 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1386/2804] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:20.511505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:20.512969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:20.569390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cdbad-4c70-4dcf-883d-42da1c7125ba admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:20.573719 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7c439-104a-4d39-abee-752cbc32ff94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:20.574390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c439-104a-4d39-abee-752cbc32ff94 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:20.574673 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c439-104a-4d39-abee-752cbc32ff94 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:20.574947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c439-104a-4d39-abee-752cbc32ff94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:20.583891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7c439-104a-4d39-abee-752cbc32ff94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:20.585301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7c439-104a-4d39-abee-752cbc32ff94 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:20.594898 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1419/2805] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:20.985680 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0e18f-e22d-4b9c-9bb8-3d7063a6cd41 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:20.987167 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d3f0e18f-e22d-4b9c-9bb8-3d7063a6cd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:21.000197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0e18f-e22d-4b9c-9bb8-3d7063a6cd41 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:21.005369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:21.006117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:21.006456 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:21.006802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] PATH_INFO: `/v3/projects/c141d6bc5e7e47e7b5b93ae029597651` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:21.009561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c141d6bc5e7e47e7b5b93ae029597651)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:21.010615 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:21.099137 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7c439-104a-4d39-abee-752cbc32ff94 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:21.108826 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7c439-104a-4d39-abee-752cbc32ff94 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:21.123395 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c141d6bc5e7e47e7b5b93ae029597651', 'request_id': 'req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.159640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:21.160332 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:21.160621 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:21.160900 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:21.162694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:21.193261 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c141d6bc5e7e47e7b5b93ae029597651', 'request_id': 'req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.199573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c141d6bc5e7e47e7b5b93ae029597651', 'request_id': 'req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.204996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:21.210969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] The token cache is being invalidate because project c141d6bc5e7e47e7b5b93ae029597651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:21.211844 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.212543 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.214415 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1387/2806] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:20 2022] DELETE /identity/v3/projects/c141d6bc5e7e47e7b5b93ae029597651 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:21.218087 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:21.219410 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:21.228952 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:21.230604 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:21.240740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1420/2807] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:21 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 25 06:08:21.266188 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:21.298570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bbbfbd-35c1-4d9c-a462-6eac8e8138de admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:21.303516 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:21.304512 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:21.305461 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:21.305805 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] PATH_INFO: `/v3/users/9bf683bf9ea545928e7fcf2cf825dec0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:21.307012 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf683bf9ea545928e7fcf2cf825dec0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:21.308957 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:21.518002 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf683bf9ea545928e7fcf2cf825dec0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.590254 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf683bf9ea545928e7fcf2cf825dec0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.591322 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf683bf9ea545928e7fcf2cf825dec0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.599372 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf683bf9ea545928e7fcf2cf825dec0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.600033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-137a46f3-86e9-40cc-bd0d-3a98158aa442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf683bf9ea545928e7fcf2cf825dec0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:21.620721 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1388/2808] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:21 2022] DELETE /identity/v3/users/9bf683bf9ea545928e7fcf2cf825dec0 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:08:22.008615 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:22.021227 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dbf99-4cfb-40f7-978e-99b696e819bc service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:22.091242 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:22.092138 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.092438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.092718 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] PATH_INFO: `/v3/projects/6821b2b15b7741fe84a66ca5a133b51d` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.094066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6821b2b15b7741fe84a66ca5a133b51d)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:22.096248 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:22.207367 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6821b2b15b7741fe84a66ca5a133b51d', 'request_id': 'req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:22.296444 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6821b2b15b7741fe84a66ca5a133b51d', 'request_id': 'req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:22.302530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6821b2b15b7741fe84a66ca5a133b51d', 'request_id': 'req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:22.317657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] The token cache is being invalidate because project 6821b2b15b7741fe84a66ca5a133b51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86792) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:22.318193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:22.318926 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-4a9b8f9e-14b1-489a-8191-b8ec5f6b254d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:22.322516 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1421/2809] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:22 2022] DELETE /identity/v3/projects/6821b2b15b7741fe84a66ca5a133b51d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:22.327117 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0763f-9ec1-4f6e-8700-15191275544a None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.327454 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0763f-9ec1-4f6e-8700-15191275544a None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.327810 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0763f-9ec1-4f6e-8700-15191275544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.354044 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4ba0763f-9ec1-4f6e-8700-15191275544a None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:22.355867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4ba0763f-9ec1-4f6e-8700-15191275544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:22.423520 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1389/2810] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:22.433473 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a9fd-9e83-45a5-8e62-712cb262fd20 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.433905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a9fd-9e83-45a5-8e62-712cb262fd20 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.434047 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42a9fd-9e83-45a5-8e62-712cb262fd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.454638 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-2a42a9fd-9e83-45a5-8e62-712cb262fd20 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:22.455891 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-2a42a9fd-9e83-45a5-8e62-712cb262fd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:22.464736 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1422/2811] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:22.468950 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0763f-9ec1-4f6e-8700-15191275544a None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:22.479414 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0763f-9ec1-4f6e-8700-15191275544a None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:22.483992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:22.484692 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.484971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.488463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] PATH_INFO: `/v3/domains` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.488463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86791) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:08:22.488463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:22.488463 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:22.500542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1390/2812] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 25 06:08:22.504889 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a9fd-9e83-45a5-8e62-712cb262fd20 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:22.516252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42a9fd-9e83-45a5-8e62-712cb262fd20 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:22.528348 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d886f-5833-47c2-8956-ad88e9ab708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:22.529129 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d886f-5833-47c2-8956-ad88e9ab708c admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.529434 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d886f-5833-47c2-8956-ad88e9ab708c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.529701 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d886f-5833-47c2-8956-ad88e9ab708c admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.530403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d886f-5833-47c2-8956-ad88e9ab708c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:22.532321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d886f-5833-47c2-8956-ad88e9ab708c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:22.625568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1423/2813] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:22 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:08:22.629827 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:22.640881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e40cb-8fe3-4ada-8654-8d0b0502bb16 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:22.683876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:22.701469 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cbed6-9e0b-4b08-9d91-20753d7c08a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:22.702155 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cbed6-9e0b-4b08-9d91-20753d7c08a2 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.703063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cbed6-9e0b-4b08-9d91-20753d7c08a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.703063 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17cbed6-9e0b-4b08-9d91-20753d7c08a2 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.703601 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cbed6-9e0b-4b08-9d91-20753d7c08a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:22.705648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17cbed6-9e0b-4b08-9d91-20753d7c08a2 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:22.801390 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1391/2814] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:22 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:22.806182 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d886f-5833-47c2-8956-ad88e9ab708c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:22.816821 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d886f-5833-47c2-8956-ad88e9ab708c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:22.822510 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ab043-23a3-4d07-82c1-32c1047c7e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:22.823183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ab043-23a3-4d07-82c1-32c1047c7e5c admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.823467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ab043-23a3-4d07-82c1-32c1047c7e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.823744 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ab043-23a3-4d07-82c1-32c1047c7e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.824543 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ab043-23a3-4d07-82c1-32c1047c7e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:22.826253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ab043-23a3-4d07-82c1-32c1047c7e5c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:22.837853 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1424/2815] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:08:22.842060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cbed6-9e0b-4b08-9d91-20753d7c08a2 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:22.852285 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17cbed6-9e0b-4b08-9d91-20753d7c08a2 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:22.856948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147d5e9-941a-4125-b741-6c97f8a02188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:22.860832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d5e9-941a-4125-b741-6c97f8a02188 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.860832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d5e9-941a-4125-b741-6c97f8a02188 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.860832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147d5e9-941a-4125-b741-6c97f8a02188 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.860832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147d5e9-941a-4125-b741-6c97f8a02188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:22.860832 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147d5e9-941a-4125-b741-6c97f8a02188 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:22.871708 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1392/2816] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:22.875945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ab043-23a3-4d07-82c1-32c1047c7e5c admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:22.887112 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ab043-23a3-4d07-82c1-32c1047c7e5c admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:22.892187 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c96ba-6667-4d11-bc7e-e752fc4fc190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:22.893106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c96ba-6667-4d11-bc7e-e752fc4fc190 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:22.893390 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c96ba-6667-4d11-bc7e-e752fc4fc190 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:22.893667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c96ba-6667-4d11-bc7e-e752fc4fc190 admin admin] PATH_INFO: `/v3/projects/464bb01a4b2b4bdfaa2d03892a15c6f4/users/da5de94173a34edd8421f5e8c470e60d/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:22.910014 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c96ba-6667-4d11-bc7e-e752fc4fc190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=464bb01a4b2b4bdfaa2d03892a15c6f4, user_id=da5de94173a34edd8421f5e8c470e60d, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:22.912729 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c96ba-6667-4d11-bc7e-e752fc4fc190 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:23.005752 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1425/2817] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:22 2022] PUT /identity/v3/projects/464bb01a4b2b4bdfaa2d03892a15c6f4/users/da5de94173a34edd8421f5e8c470e60d/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:23.017460 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f151ed-55d1-4bab-b619-1959e01b6ccb None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:23.018001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f151ed-55d1-4bab-b619-1959e01b6ccb None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:23.018282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f151ed-55d1-4bab-b619-1959e01b6ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:23.045697 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-d4f151ed-55d1-4bab-b619-1959e01b6ccb None None] MFA Rules not processed for user `da5de94173a34edd8421f5e8c470e60d`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:23.047257 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d4f151ed-55d1-4bab-b619-1959e01b6ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:23.114154 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1393/2818] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:23.120148 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c96ba-6667-4d11-bc7e-e752fc4fc190 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:23.124863 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:23.183859 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c96ba-6667-4d11-bc7e-e752fc4fc190 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:23.189066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e639e39-fa41-471d-80db-97ba1d5f922b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:23.189735 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e639e39-fa41-471d-80db-97ba1d5f922b admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:23.190019 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e639e39-fa41-471d-80db-97ba1d5f922b admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:23.190297 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e639e39-fa41-471d-80db-97ba1d5f922b admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:23.190997 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e639e39-fa41-471d-80db-97ba1d5f922b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:23.192981 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e639e39-fa41-471d-80db-97ba1d5f922b admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:23.276335 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1426/2819] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:23 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:23.280491 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f151ed-55d1-4bab-b619-1959e01b6ccb None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:23.291502 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f151ed-55d1-4bab-b619-1959e01b6ccb None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:23.351054 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc1de7-930f-4a64-8b90-ca45fedf82b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:23.351705 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc1de7-930f-4a64-8b90-ca45fedf82b8 admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:23.352259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc1de7-930f-4a64-8b90-ca45fedf82b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:23.352259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc1de7-930f-4a64-8b90-ca45fedf82b8 admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:23.352971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc1de7-930f-4a64-8b90-ca45fedf82b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:23.354932 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc1de7-930f-4a64-8b90-ca45fedf82b8 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:23.473216 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1394/2820] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:23 2022] POST /identity/v3/users => generated 292 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:23.478801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e639e39-fa41-471d-80db-97ba1d5f922b admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:23.494440 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e639e39-fa41-471d-80db-97ba1d5f922b admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:23.496901 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d74118-db37-46d9-80e1-e031d3553e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:23.497825 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74118-db37-46d9-80e1-e031d3553e2e admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:23.498152 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74118-db37-46d9-80e1-e031d3553e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:23.499132 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d74118-db37-46d9-80e1-e031d3553e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:23.499887 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d74118-db37-46d9-80e1-e031d3553e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:23.501657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d74118-db37-46d9-80e1-e031d3553e2e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:23.514033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1427/2821] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:23.518274 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc1de7-930f-4a64-8b90-ca45fedf82b8 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:23.532262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc1de7-930f-4a64-8b90-ca45fedf82b8 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:23.537652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed4f89-bb08-4804-ab11-0354146fbc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:23.538301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed4f89-bb08-4804-ab11-0354146fbc74 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:23.538715 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed4f89-bb08-4804-ab11-0354146fbc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:23.539092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed4f89-bb08-4804-ab11-0354146fbc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:23.540675 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed4f89-bb08-4804-ab11-0354146fbc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:23.542378 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed4f89-bb08-4804-ab11-0354146fbc74 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:23.556850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1395/2822] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:23.558436 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d74118-db37-46d9-80e1-e031d3553e2e admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:23.571174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d74118-db37-46d9-80e1-e031d3553e2e admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:23.577183 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac030d3-055c-41ee-a3ef-d6859f7b9ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:23.585164 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac030d3-055c-41ee-a3ef-d6859f7b9ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:23.586547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac030d3-055c-41ee-a3ef-d6859f7b9ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:23.587286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac030d3-055c-41ee-a3ef-d6859f7b9ad4 admin admin] PATH_INFO: `/v3/projects/69e56ce51dde4fcbb31bd289913a203c/users/63d379dc8b1942b29f98e3c85bb272b3/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:23.614812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac030d3-055c-41ee-a3ef-d6859f7b9ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e56ce51dde4fcbb31bd289913a203c, user_id=63d379dc8b1942b29f98e3c85bb272b3, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:23.618327 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac030d3-055c-41ee-a3ef-d6859f7b9ad4 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:23.693984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1428/2823] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:23 2022] PUT /identity/v3/projects/69e56ce51dde4fcbb31bd289913a203c/users/63d379dc8b1942b29f98e3c85bb272b3/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:23.705884 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a42e-778c-420f-a4c6-8b5ea9c55380 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:23.706431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a42e-778c-420f-a4c6-8b5ea9c55380 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:23.706431 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4a42e-778c-420f-a4c6-8b5ea9c55380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:23.738267 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-38c4a42e-778c-420f-a4c6-8b5ea9c55380 None None] MFA Rules not processed for user `63d379dc8b1942b29f98e3c85bb272b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:23.739519 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-38c4a42e-778c-420f-a4c6-8b5ea9c55380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:23.769357 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:23.827542 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1396/2824] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:23.891201 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac030d3-055c-41ee-a3ef-d6859f7b9ad4 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:23.963805 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac030d3-055c-41ee-a3ef-d6859f7b9ad4 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:23.968858 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49698e-0405-49bf-9470-2d10ef111e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:23.974362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49698e-0405-49bf-9470-2d10ef111e5f admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:23.974362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49698e-0405-49bf-9470-2d10ef111e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:23.974362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a49698e-0405-49bf-9470-2d10ef111e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:23.974362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49698e-0405-49bf-9470-2d10ef111e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:23.974362 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a49698e-0405-49bf-9470-2d10ef111e5f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:24.053333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1429/2825] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:23 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:24.057926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4a42e-778c-420f-a4c6-8b5ea9c55380 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:24.069275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4a42e-778c-420f-a4c6-8b5ea9c55380 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:24.142425 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d36b0-d80f-42df-8767-690ec09519cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:24.143209 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d36b0-d80f-42df-8767-690ec09519cc admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:24.143621 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d36b0-d80f-42df-8767-690ec09519cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:24.144331 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d36b0-d80f-42df-8767-690ec09519cc admin admin] PATH_INFO: `/v3/users` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:24.145062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d36b0-d80f-42df-8767-690ec09519cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:24.147112 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d36b0-d80f-42df-8767-690ec09519cc admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:24.239286 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1397/2826] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:24 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:24.240507 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49698e-0405-49bf-9470-2d10ef111e5f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:24.252177 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a49698e-0405-49bf-9470-2d10ef111e5f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:24.256931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352263e-4bed-4975-9892-19157fc204e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:24.261141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352263e-4bed-4975-9892-19157fc204e0 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:24.261141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352263e-4bed-4975-9892-19157fc204e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:24.261141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352263e-4bed-4975-9892-19157fc204e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:24.261141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352263e-4bed-4975-9892-19157fc204e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:24.261526 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f352263e-4bed-4975-9892-19157fc204e0 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:24.275708 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1430/2827] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:24.276993 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d36b0-d80f-42df-8767-690ec09519cc admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:24.287955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d36b0-d80f-42df-8767-690ec09519cc admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:24.293047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94cd3d-6fbc-4e19-95da-ca7e4ce8f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:24.293944 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94cd3d-6fbc-4e19-95da-ca7e4ce8f9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:24.294233 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94cd3d-6fbc-4e19-95da-ca7e4ce8f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:24.294504 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94cd3d-6fbc-4e19-95da-ca7e4ce8f9a3 admin admin] PATH_INFO: `/v3/projects/da8a388cf7c742c397db5df2af43f062/users/cb9b4624f9e54276972cec7692ffc8ac/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:24.311259 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94cd3d-6fbc-4e19-95da-ca7e4ce8f9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8a388cf7c742c397db5df2af43f062, user_id=cb9b4624f9e54276972cec7692ffc8ac, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:24.313955 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94cd3d-6fbc-4e19-95da-ca7e4ce8f9a3 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:24.436215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1398/2828] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:24 2022] PUT /identity/v3/projects/da8a388cf7c742c397db5df2af43f062/users/cb9b4624f9e54276972cec7692ffc8ac/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:24.440700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352263e-4bed-4975-9892-19157fc204e0 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:24.454004 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352263e-4bed-4975-9892-19157fc204e0 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:24.482068 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:24.518818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac0a89-0e82-40bb-909c-b35a4ae502db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:24.519666 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac0a89-0e82-40bb-909c-b35a4ae502db admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:24.519947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac0a89-0e82-40bb-909c-b35a4ae502db admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:24.520273 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac0a89-0e82-40bb-909c-b35a4ae502db admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:24.520990 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac0a89-0e82-40bb-909c-b35a4ae502db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:24.522730 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac0a89-0e82-40bb-909c-b35a4ae502db admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:24.536888 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1431/2829] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:24.538049 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94cd3d-6fbc-4e19-95da-ca7e4ce8f9a3 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:24.549328 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94cd3d-6fbc-4e19-95da-ca7e4ce8f9a3 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:24.554073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:24.558368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:24.558368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:24.558368 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] PATH_INFO: `/v3/projects/da8a388cf7c742c397db5df2af43f062/users/cb9b4624f9e54276972cec7692ffc8ac/roles/e9f347a638824dd09ef0cc73e9bb2e5b` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:24.558690 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8a388cf7c742c397db5df2af43f062, user_id=cb9b4624f9e54276972cec7692ffc8ac, role_id=e9f347a638824dd09ef0cc73e9bb2e5b)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:24.561228 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:24.572339 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1399/2830] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:24 2022] PUT /identity/v3/projects/da8a388cf7c742c397db5df2af43f062/users/cb9b4624f9e54276972cec7692ffc8ac/roles/e9f347a638824dd09ef0cc73e9bb2e5b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:08:24.582533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa57a5-4a84-4b3b-a286-d9afdfa5b971 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:24.582823 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa57a5-4a84-4b3b-a286-d9afdfa5b971 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:24.583101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa57a5-4a84-4b3b-a286-d9afdfa5b971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:24.609761 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-a2fa57a5-4a84-4b3b-a286-d9afdfa5b971 None None] MFA Rules not processed for user `cb9b4624f9e54276972cec7692ffc8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:24.610989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a2fa57a5-4a84-4b3b-a286-d9afdfa5b971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:24.680931 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1432/2831] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 25 06:08:24.692264 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:24.693956 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:24.755376 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0181b3b-98eb-4fa2-a101-71987a13d528 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:24.759846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a95559-422a-49d1-8e7e-4000d99747b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:24.760610 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a95559-422a-49d1-8e7e-4000d99747b2 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:24.760899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a95559-422a-49d1-8e7e-4000d99747b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:24.761180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a95559-422a-49d1-8e7e-4000d99747b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:24.786074 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:24.818046 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a95559-422a-49d1-8e7e-4000d99747b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:24.819478 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a95559-422a-49d1-8e7e-4000d99747b2 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:24.829529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1400/2832] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:24.896556 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa57a5-4a84-4b3b-a286-d9afdfa5b971 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:24.897968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-a2fa57a5-4a84-4b3b-a286-d9afdfa5b971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:24.967286 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fa57a5-4a84-4b3b-a286-d9afdfa5b971 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:24.971728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:24.972637 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:24.972919 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:24.973193 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] PATH_INFO: `/v3/users/a34e3ccda41449de98171968bcbc2798` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:24.974406 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a34e3ccda41449de98171968bcbc2798)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:24.976394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:25.156212 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a34e3ccda41449de98171968bcbc2798'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:25.158746 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:25.216815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a34e3ccda41449de98171968bcbc2798'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:25.217515 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a34e3ccda41449de98171968bcbc2798'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:25.224634 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a34e3ccda41449de98171968bcbc2798'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:25.225253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a34e3ccda41449de98171968bcbc2798'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:25.241936 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1433/2833] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:24 2022] DELETE /identity/v3/users/a34e3ccda41449de98171968bcbc2798 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:25.247120 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d2a7-329e-47ad-8cb1-1c338583b1c7 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:25.247411 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d2a7-329e-47ad-8cb1-1c338583b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:25.247691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91d2a7-329e-47ad-8cb1-1c338583b1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:25.265720 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-5d91d2a7-329e-47ad-8cb1-1c338583b1c7 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:25.266930 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5d91d2a7-329e-47ad-8cb1-1c338583b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:25.325353 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1401/2834] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:25.334836 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:25.336432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:25.397657 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb86f7-991d-41d0-b5f9-caa5a94c4118 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:25.403675 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:25.403675 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:25.403675 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:25.404092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:25.413998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:25.417024 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:25.427001 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1434/2835] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:26.110725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91d2a7-329e-47ad-8cb1-1c338583b1c7 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:26.112524 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-5d91d2a7-329e-47ad-8cb1-1c338583b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:26.125230 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91d2a7-329e-47ad-8cb1-1c338583b1c7 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:26.130252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:26.131175 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:26.131464 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:26.131750 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] PATH_INFO: `/v3/projects/5f50cc3ec1d646a4b6ac97c0e11e8ea5` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:26.133740 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f50cc3ec1d646a4b6ac97c0e11e8ea5)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:26.135564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:26.250587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f50cc3ec1d646a4b6ac97c0e11e8ea5', 'request_id': 'req-1fd721c7-631a-4ccc-91cf-331b1d8b7326'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:26.312833 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f50cc3ec1d646a4b6ac97c0e11e8ea5', 'request_id': 'req-1fd721c7-631a-4ccc-91cf-331b1d8b7326'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:26.318547 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f50cc3ec1d646a4b6ac97c0e11e8ea5', 'request_id': 'req-1fd721c7-631a-4ccc-91cf-331b1d8b7326'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:26.330330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] The token cache is being invalidate because project 5f50cc3ec1d646a4b6ac97c0e11e8ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:26.330620 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:26.331275 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:26.332860 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1402/2836] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:26 2022] DELETE /identity/v3/projects/5f50cc3ec1d646a4b6ac97c0e11e8ea5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:26.706907 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:26.708358 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:26.766093 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c279c9-14a6-485a-b038-444b5318e180 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:26.813546 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:26.814190 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:26.814472 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:26.814747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:26.816540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:26.855583 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:26.872916 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:26.874383 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:26.883355 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1435/2837] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:28.311629 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:28.313055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:28.325473 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd721c7-631a-4ccc-91cf-331b1d8b7326 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:28.329029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c665bd6-b66b-4e75-b28f-f065e1516e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:28.329694 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665bd6-b66b-4e75-b28f-f065e1516e54 service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:28.329974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665bd6-b66b-4e75-b28f-f065e1516e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:28.330248 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c665bd6-b66b-4e75-b28f-f065e1516e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:28.331939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-8c665bd6-b66b-4e75-b28f-f065e1516e54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:28.377180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:28.394969 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c665bd6-b66b-4e75-b28f-f065e1516e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:28.396533 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c665bd6-b66b-4e75-b28f-f065e1516e54 service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:28.406622 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1403/2838] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:32.811045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:32.812574 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:32.876042 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c21311-0d86-44d4-800d-865bed24b744 service cinder] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:32.880618 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:32.881495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:32.881765 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:32.882044 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] PATH_INFO: `/v3/users/da5de94173a34edd8421f5e8c470e60d` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:32.883782 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5de94173a34edd8421f5e8c470e60d)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:32.885739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:33.088601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5de94173a34edd8421f5e8c470e60d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:33.159070 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5de94173a34edd8421f5e8c470e60d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:33.159694 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5de94173a34edd8421f5e8c470e60d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:33.167756 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5de94173a34edd8421f5e8c470e60d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:33.168902 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5de94173a34edd8421f5e8c470e60d'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:33.187458 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1436/2839] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:32 2022] DELETE /identity/v3/users/da5de94173a34edd8421f5e8c470e60d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:33.192033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d95ad-b8d8-4bcf-9076-2eb313a14634 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:33.192574 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d95ad-b8d8-4bcf-9076-2eb313a14634 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:33.192674 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d95ad-b8d8-4bcf-9076-2eb313a14634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:33.211243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-499d95ad-b8d8-4bcf-9076-2eb313a14634 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:33.212471 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-499d95ad-b8d8-4bcf-9076-2eb313a14634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:33.275351 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1404/2840] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:33.285141 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:33.286563 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:33.347530 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029818b-b7dd-4eae-b1d8-b5a8e5c7bcb4 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:33.351239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e80c90-d4dc-428b-9f27-e80151895890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:33.352325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e80c90-d4dc-428b-9f27-e80151895890 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:33.352325 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e80c90-d4dc-428b-9f27-e80151895890 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:33.352595 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e80c90-d4dc-428b-9f27-e80151895890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:33.362449 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e80c90-d4dc-428b-9f27-e80151895890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:33.363896 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e80c90-d4dc-428b-9f27-e80151895890 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:33.375113 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1437/2841] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:33.900332 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d95ad-b8d8-4bcf-9076-2eb313a14634 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:33.901807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-499d95ad-b8d8-4bcf-9076-2eb313a14634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:33.914830 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d95ad-b8d8-4bcf-9076-2eb313a14634 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:33.919915 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:33.920856 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:33.921148 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:33.921895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] PATH_INFO: `/v3/projects/464bb01a4b2b4bdfaa2d03892a15c6f4` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:33.922804 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=464bb01a4b2b4bdfaa2d03892a15c6f4)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:33.924772 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:34.013090 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '464bb01a4b2b4bdfaa2d03892a15c6f4', 'request_id': 'req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.080561 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '464bb01a4b2b4bdfaa2d03892a15c6f4', 'request_id': 'req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.088062 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '464bb01a4b2b4bdfaa2d03892a15c6f4', 'request_id': 'req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.099656 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] The token cache is being invalidate because project 464bb01a4b2b4bdfaa2d03892a15c6f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:34.099946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.100638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.102394 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1405/2842] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:33 2022] DELETE /identity/v3/projects/464bb01a4b2b4bdfaa2d03892a15c6f4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:34.106418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e80c90-d4dc-428b-9f27-e80151895890 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:34.111792 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:34.118246 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e80c90-d4dc-428b-9f27-e80151895890 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:34.178432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:34.179293 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:34.180162 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:34.180598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] PATH_INFO: `/v3/users/63d379dc8b1942b29f98e3c85bb272b3` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:34.181995 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d379dc8b1942b29f98e3c85bb272b3)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:34.183965 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:34.403163 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d379dc8b1942b29f98e3c85bb272b3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.474982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d379dc8b1942b29f98e3c85bb272b3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.475636 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d379dc8b1942b29f98e3c85bb272b3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.483820 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d379dc8b1942b29f98e3c85bb272b3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.484497 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d379dc8b1942b29f98e3c85bb272b3'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:34.502985 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1438/2843] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:34 2022] DELETE /identity/v3/users/63d379dc8b1942b29f98e3c85bb272b3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:08:34.905281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:34.906781 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:34.921607 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a4bd1-6132-4db3-8fd7-7dd3e7e70a1e admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:34.981189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:34.981981 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:34.982258 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:34.982529 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] PATH_INFO: `/v3/projects/69e56ce51dde4fcbb31bd289913a203c` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:34.983725 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e56ce51dde4fcbb31bd289913a203c)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:34.985654 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:34.994531 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:35.109707 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e56ce51dde4fcbb31bd289913a203c', 'request_id': 'req-7dabf5e5-1a2d-40c6-a639-a82047de2a82'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.171722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e56ce51dde4fcbb31bd289913a203c', 'request_id': 'req-7dabf5e5-1a2d-40c6-a639-a82047de2a82'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.177440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e56ce51dde4fcbb31bd289913a203c', 'request_id': 'req-7dabf5e5-1a2d-40c6-a639-a82047de2a82'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.189183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] The token cache is being invalidate because project 69e56ce51dde4fcbb31bd289913a203c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:35.189472 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.190103 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.191633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1406/2844] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:34 2022] DELETE /identity/v3/projects/69e56ce51dde4fcbb31bd289913a203c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:35.195785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:35.199961 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:35.209747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f78ce-34bd-41ab-9f25-9b2bd6db207f admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:35.267200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:35.267989 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:35.268310 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:35.268593 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] PATH_INFO: `/v3/users/cb9b4624f9e54276972cec7692ffc8ac` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:35.269778 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9b4624f9e54276972cec7692ffc8ac)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:35.271727 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:35.294388 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:35.454582 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9b4624f9e54276972cec7692ffc8ac'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.525414 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9b4624f9e54276972cec7692ffc8ac'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.525958 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9b4624f9e54276972cec7692ffc8ac'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.533078 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9b4624f9e54276972cec7692ffc8ac'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.533697 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-9ef249b2-611d-4bc0-ab3a-198dac17ea5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9b4624f9e54276972cec7692ffc8ac'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:35.551037 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1439/2845] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:35 2022] DELETE /identity/v3/users/cb9b4624f9e54276972cec7692ffc8ac => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:08:35.955001 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:35.956758 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:35.970812 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabf5e5-1a2d-40c6-a639-a82047de2a82 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:36.028982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:36.029852 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.030139 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.030440 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] PATH_INFO: `/v3/projects/da8a388cf7c742c397db5df2af43f062` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.031650 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8a388cf7c742c397db5df2af43f062)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:36.033693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:36.130618 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8a388cf7c742c397db5df2af43f062', 'request_id': 'req-64ec05a0-9210-4f09-8c37-d6242109ae04'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:36.210068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8a388cf7c742c397db5df2af43f062', 'request_id': 'req-64ec05a0-9210-4f09-8c37-d6242109ae04'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:36.216131 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8a388cf7c742c397db5df2af43f062', 'request_id': 'req-64ec05a0-9210-4f09-8c37-d6242109ae04'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:36.227631 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] The token cache is being invalidate because project da8a388cf7c742c397db5df2af43f062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:36.227919 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:36.228587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-64ec05a0-9210-4f09-8c37-d6242109ae04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:36.230130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1407/2846] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:35 2022] DELETE /identity/v3/projects/da8a388cf7c742c397db5df2af43f062 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:08:36.235315 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad840-68f7-4bd1-b570-39c054422397 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.235812 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad840-68f7-4bd1-b570-39c054422397 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.235927 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1ad840-68f7-4bd1-b570-39c054422397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.255011 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-4c1ad840-68f7-4bd1-b570-39c054422397 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:36.256237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-4c1ad840-68f7-4bd1-b570-39c054422397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:36.315495 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1440/2847] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:36.324211 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bca0f-bc3d-4d68-b524-214dd8e6a34b None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.324505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bca0f-bc3d-4d68-b524-214dd8e6a34b None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.324782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bca0f-bc3d-4d68-b524-214dd8e6a34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.344170 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-f42bca0f-bc3d-4d68-b524-214dd8e6a34b None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:36.345391 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-f42bca0f-bc3d-4d68-b524-214dd8e6a34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:36.353819 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1408/2848] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:36.357984 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad840-68f7-4bd1-b570-39c054422397 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:36.368197 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1ad840-68f7-4bd1-b570-39c054422397 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:36.372760 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:36.373411 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.373696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.373976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.374706 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:08:36.375013 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:36.376667 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:36.388692 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1441/2849] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:08:36.392761 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bca0f-bc3d-4d68-b524-214dd8e6a34b None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:36.402310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bca0f-bc3d-4d68-b524-214dd8e6a34b None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:36.406942 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59538dcb-ea95-44fc-9a74-d9e5e4c692df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:36.407613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59538dcb-ea95-44fc-9a74-d9e5e4c692df admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.407897 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59538dcb-ea95-44fc-9a74-d9e5e4c692df admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.408281 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59538dcb-ea95-44fc-9a74-d9e5e4c692df admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.408967 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59538dcb-ea95-44fc-9a74-d9e5e4c692df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:36.410874 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59538dcb-ea95-44fc-9a74-d9e5e4c692df admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:36.519693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:36.527319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1409/2850] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:36 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:36.531433 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:36.542715 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd664c-6648-40e6-b210-23311868530d admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:36.597832 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0ea4-79f2-4088-8799-fa8e55a10435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:36.598511 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0ea4-79f2-4088-8799-fa8e55a10435 admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.598800 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0ea4-79f2-4088-8799-fa8e55a10435 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.599085 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0ea4-79f2-4088-8799-fa8e55a10435 admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.599779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b0ea4-79f2-4088-8799-fa8e55a10435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:36.602351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b0ea4-79f2-4088-8799-fa8e55a10435 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:36.669095 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1442/2851] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:36 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 25 06:08:36.673219 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59538dcb-ea95-44fc-9a74-d9e5e4c692df admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:36.683876 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59538dcb-ea95-44fc-9a74-d9e5e4c692df admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:36.688538 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6c73-09a7-498e-a427-b075c94505af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:36.689225 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6c73-09a7-498e-a427-b075c94505af admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.689513 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6c73-09a7-498e-a427-b075c94505af admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.689789 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6c73-09a7-498e-a427-b075c94505af admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.690545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c6c73-09a7-498e-a427-b075c94505af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:36.692164 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c6c73-09a7-498e-a427-b075c94505af admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:36.703141 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1410/2852] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:08:36.706970 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0ea4-79f2-4088-8799-fa8e55a10435 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:36.717504 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0ea4-79f2-4088-8799-fa8e55a10435 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:36.722060 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f773cf3-44c2-4657-ac0b-cbce2707906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:36.722724 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f773cf3-44c2-4657-ac0b-cbce2707906a admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.723000 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f773cf3-44c2-4657-ac0b-cbce2707906a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.723280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f773cf3-44c2-4657-ac0b-cbce2707906a admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.724005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f773cf3-44c2-4657-ac0b-cbce2707906a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:36.725632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f773cf3-44c2-4657-ac0b-cbce2707906a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:36.735537 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1443/2853] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:36.739579 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6c73-09a7-498e-a427-b075c94505af admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:36.749739 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6c73-09a7-498e-a427-b075c94505af admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:36.754449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:36.755340 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.755624 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.755900 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] PATH_INFO: `/v3/projects/e2cc955ec5e44625975f602eca4f45eb/users/4c7ebaf5af884ea7b4c966a8546844e8/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.771759 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2cc955ec5e44625975f602eca4f45eb, user_id=4c7ebaf5af884ea7b4c966a8546844e8, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:36.775626 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:36.841080 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1411/2854] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:36 2022] PUT /identity/v3/projects/e2cc955ec5e44625975f602eca4f45eb/users/4c7ebaf5af884ea7b4c966a8546844e8/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:36.852490 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d643-508c-405e-883b-8b86a445fca3 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:36.852785 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d643-508c-405e-883b-8b86a445fca3 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:36.853066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4d643-508c-405e-883b-8b86a445fca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:36.865698 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:36.879747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-dea4d643-508c-405e-883b-8b86a445fca3 None None] MFA Rules not processed for user `4c7ebaf5af884ea7b4c966a8546844e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:36.881012 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-dea4d643-508c-405e-883b-8b86a445fca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:36.946006 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1444/2855] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:36.956926 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:36.958695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:37.014948 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ec568-b396-4564-9224-d4a7a0aef409 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:37.018671 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7ab4fc-d111-4f63-a355-ca8ad8fb6ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:37.019330 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ab4fc-d111-4f63-a355-ca8ad8fb6ddc service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:37.019616 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ab4fc-d111-4f63-a355-ca8ad8fb6ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:37.022055 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7ab4fc-d111-4f63-a355-ca8ad8fb6ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:37.029409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ab4fc-d111-4f63-a355-ca8ad8fb6ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:37.031058 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7ab4fc-d111-4f63-a355-ca8ad8fb6ddc service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:37.040479 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1412/2856] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:37.064403 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4d643-508c-405e-883b-8b86a445fca3 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:37.127171 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4d643-508c-405e-883b-8b86a445fca3 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:37.131632 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:37.132527 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:37.133457 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:37.133740 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] PATH_INFO: `/v3/users/4c7ebaf5af884ea7b4c966a8546844e8` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:37.134929 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c7ebaf5af884ea7b4c966a8546844e8)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:37.136875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:37.276967 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c7ebaf5af884ea7b4c966a8546844e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:37.320420 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c7ebaf5af884ea7b4c966a8546844e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:37.321252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c7ebaf5af884ea7b4c966a8546844e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:37.329288 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c7ebaf5af884ea7b4c966a8546844e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:37.329969 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c7ebaf5af884ea7b4c966a8546844e8'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:37.348781 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1445/2857] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:37 2022] DELETE /identity/v3/users/4c7ebaf5af884ea7b4c966a8546844e8 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:37.354152 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18677e-8611-43a5-accd-be322d9b9dec None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:37.354449 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18677e-8611-43a5-accd-be322d9b9dec None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:37.354730 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18677e-8611-43a5-accd-be322d9b9dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:37.373455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-4a18677e-8611-43a5-accd-be322d9b9dec None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:37.374652 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4a18677e-8611-43a5-accd-be322d9b9dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:37.434341 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1413/2858] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:37.443910 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:37.445400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:37.464613 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:37.503533 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834b13c9-708e-4579-a654-325d4a136a84 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:37.507178 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79fb08-c6d6-42d9-804b-2dfe0ab3305f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:37.507982 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79fb08-c6d6-42d9-804b-2dfe0ab3305f service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:37.508610 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79fb08-c6d6-42d9-804b-2dfe0ab3305f service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:37.508922 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79fb08-c6d6-42d9-804b-2dfe0ab3305f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:37.518226 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79fb08-c6d6-42d9-804b-2dfe0ab3305f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:37.519649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79fb08-c6d6-42d9-804b-2dfe0ab3305f service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:37.530829 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1446/2859] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:37.916462 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18677e-8611-43a5-accd-be322d9b9dec None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:37.918118 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4a18677e-8611-43a5-accd-be322d9b9dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:37.938989 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18677e-8611-43a5-accd-be322d9b9dec None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:37.947821 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:37.949072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:37.949505 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:37.950014 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] PATH_INFO: `/v3/projects/e2cc955ec5e44625975f602eca4f45eb` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:37.951344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2cc955ec5e44625975f602eca4f45eb)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:37.953560 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:38.042374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2cc955ec5e44625975f602eca4f45eb', 'request_id': 'req-321185df-0eac-45a7-b39f-a26005ed26f0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:38.097545 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2cc955ec5e44625975f602eca4f45eb', 'request_id': 'req-321185df-0eac-45a7-b39f-a26005ed26f0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:38.102180 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:38.104284 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2cc955ec5e44625975f602eca4f45eb', 'request_id': 'req-321185df-0eac-45a7-b39f-a26005ed26f0'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:38.115221 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] The token cache is being invalidate because project e2cc955ec5e44625975f602eca4f45eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:08:38.115507 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:38.116150 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-321185df-0eac-45a7-b39f-a26005ed26f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:38.118702 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1414/2860] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:08:37 2022] DELETE /identity/v3/projects/e2cc955ec5e44625975f602eca4f45eb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:38.123672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd14d67-9dd3-4b62-a649-43ce1e473b20 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.124021 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd14d67-9dd3-4b62-a649-43ce1e473b20 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.130280 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd14d67-9dd3-4b62-a649-43ce1e473b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.142562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-abd14d67-9dd3-4b62-a649-43ce1e473b20 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:38.143747 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-abd14d67-9dd3-4b62-a649-43ce1e473b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:38.203438 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1447/2861] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:38.211841 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0759d3f-32dc-4e35-9981-b80283062f1f None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.212344 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0759d3f-32dc-4e35-9981-b80283062f1f None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.212634 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0759d3f-32dc-4e35-9981-b80283062f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.230672 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-b0759d3f-32dc-4e35-9981-b80283062f1f None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:38.231881 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-b0759d3f-32dc-4e35-9981-b80283062f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:38.239767 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1415/2862] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:38.244220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd14d67-9dd3-4b62-a649-43ce1e473b20 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:38.254059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd14d67-9dd3-4b62-a649-43ce1e473b20 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:38.259245 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:38.259905 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.260239 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.260523 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] PATH_INFO: `/v3/domains` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.261256 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86792) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 25 06:08:38.261568 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:38.263169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:38.275739 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1448/2863] 10.222.0.40 () {58 vars in 1152 bytes} [Mon Jul 25 06:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 25 06:08:38.279825 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0759d3f-32dc-4e35-9981-b80283062f1f None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:38.289037 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0759d3f-32dc-4e35-9981-b80283062f1f None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:38.293712 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14651027-c65c-489a-b661-1900b56e80cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:38.294301 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14651027-c65c-489a-b661-1900b56e80cc admin admin] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.294587 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14651027-c65c-489a-b661-1900b56e80cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.294870 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14651027-c65c-489a-b661-1900b56e80cc admin admin] PATH_INFO: `/v3/projects` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.295595 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14651027-c65c-489a-b661-1900b56e80cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:38.297532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14651027-c65c-489a-b661-1900b56e80cc admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:38.407831 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1416/2864] 10.222.0.40 () {60 vars in 1152 bytes} [Mon Jul 25 06:08:38 2022] POST /identity/v3/projects => generated 391 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:08:38.411976 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:38.422640 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030ed98-3bcc-4a65-9cc6-61782b029d75 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:38.481435 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d63f8-1820-4622-8118-166478070e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:38.482110 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d63f8-1820-4622-8118-166478070e5a admin admin] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.482395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d63f8-1820-4622-8118-166478070e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.482671 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d63f8-1820-4622-8118-166478070e5a admin admin] PATH_INFO: `/v3/users` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.483356 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d63f8-1820-4622-8118-166478070e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:38.485316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d63f8-1820-4622-8118-166478070e5a admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:38.572828 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1449/2865] 10.222.0.40 () {60 vars in 1143 bytes} [Mon Jul 25 06:08:38 2022] POST /identity/v3/users => generated 296 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 25 06:08:38.577007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14651027-c65c-489a-b661-1900b56e80cc admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:38.587633 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14651027-c65c-489a-b661-1900b56e80cc admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:38.592555 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9599c9f-b99b-4cf6-8b6f-4906b9b3cc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:38.593272 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9599c9f-b99b-4cf6-8b6f-4906b9b3cc59 admin admin] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.593554 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9599c9f-b99b-4cf6-8b6f-4906b9b3cc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.593836 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9599c9f-b99b-4cf6-8b6f-4906b9b3cc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.594573 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9599c9f-b99b-4cf6-8b6f-4906b9b3cc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:38.596207 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9599c9f-b99b-4cf6-8b6f-4906b9b3cc59 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:38.607199 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1417/2866] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 25 06:08:38.611169 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d63f8-1820-4622-8118-166478070e5a admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:38.622031 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d63f8-1820-4622-8118-166478070e5a admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:38.626649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eda13b-392e-46fb-b219-01600093e03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:38.627314 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eda13b-392e-46fb-b219-01600093e03c admin admin] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.627598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eda13b-392e-46fb-b219-01600093e03c admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.627875 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eda13b-392e-46fb-b219-01600093e03c admin admin] PATH_INFO: `/v3/roles` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.629135 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eda13b-392e-46fb-b219-01600093e03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:38.630301 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eda13b-392e-46fb-b219-01600093e03c admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:38.640996 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1450/2867] 10.222.0.40 () {58 vars in 1121 bytes} [Mon Jul 25 06:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 25 06:08:38.645260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9599c9f-b99b-4cf6-8b6f-4906b9b3cc59 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:38.655766 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9599c9f-b99b-4cf6-8b6f-4906b9b3cc59 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:38.660369 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:38.661241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] REQUEST_METHOD: `PUT` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.661526 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.661803 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] PATH_INFO: `/v3/projects/03ee21bb5dad41f6b6d427201433123d/users/8902febb8a5c47fbb69de65e5cc15e1e/roles/45ad0499b0b54c309464a2ebb8a3e5d0` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.676807 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ee21bb5dad41f6b6d427201433123d, user_id=8902febb8a5c47fbb69de65e5cc15e1e, role_id=45ad0499b0b54c309464a2ebb8a3e5d0)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:38.680921 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:38.731861 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1418/2868] 10.222.0.40 () {60 vars in 1482 bytes} [Mon Jul 25 06:08:38 2022] PUT /identity/v3/projects/03ee21bb5dad41f6b6d427201433123d/users/8902febb8a5c47fbb69de65e5cc15e1e/roles/45ad0499b0b54c309464a2ebb8a3e5d0 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:38.743253 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7136f4-d810-4360-bf46-7b1521537703 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.743576 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7136f4-d810-4360-bf46-7b1521537703 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.743880 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7136f4-d810-4360-bf46-7b1521537703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.773010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-ea7136f4-d810-4360-bf46-7b1521537703 None None] MFA Rules not processed for user `8902febb8a5c47fbb69de65e5cc15e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:38.774950 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ea7136f4-d810-4360-bf46-7b1521537703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:38.786392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:38.839974 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1451/2869] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:08:38.850424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:38.852878 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:08:38.908885 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6c624-3f74-4bb8-881c-068716bfff3d admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:38.912565 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aea2625-c463-4acc-a0cc-09fab48fcf7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8a47b60be1a45f9aabc262b2d04d596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:38.913208 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea2625-c463-4acc-a0cc-09fab48fcf7f service cinder] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:38.913496 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea2625-c463-4acc-a0cc-09fab48fcf7f service cinder] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:38.913774 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aea2625-c463-4acc-a0cc-09fab48fcf7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:38.922976 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea2625-c463-4acc-a0cc-09fab48fcf7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:38.924427 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aea2625-c463-4acc-a0cc-09fab48fcf7f service cinder] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:38.933813 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1419/2870] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:08:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:08:48.837102 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=21 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:48.929243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:DELETE=1,keystone:SELECT=57,keystone:INSERT=2 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:08:59.583321 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7136f4-d810-4360-bf46-7b1521537703 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:08:59.644200 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7136f4-d810-4360-bf46-7b1521537703 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:08:59.648801 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:08:59.649906 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:59.650188 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:59.650459 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] PATH_INFO: `/v3/users/8902febb8a5c47fbb69de65e5cc15e1e` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:59.651655 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8902febb8a5c47fbb69de65e5cc15e1e)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:08:59.653607 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:08:59.830591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8902febb8a5c47fbb69de65e5cc15e1e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:59.903649 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8902febb8a5c47fbb69de65e5cc15e1e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:59.904317 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8902febb8a5c47fbb69de65e5cc15e1e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:59.911591 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8902febb8a5c47fbb69de65e5cc15e1e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:59.912267 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8902febb8a5c47fbb69de65e5cc15e1e'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:08:59.930560 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1452/2871] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:08:59 2022] DELETE /identity/v3/users/8902febb8a5c47fbb69de65e5cc15e1e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:08:59.935073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eb67a-144f-45f9-aaa8-959a58371129 None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:08:59.935546 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eb67a-144f-45f9-aaa8-959a58371129 None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:08:59.935751 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4eb67a-144f-45f9-aaa8-959a58371129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:08:59.953282 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-da4eb67a-144f-45f9-aaa8-959a58371129 None None] MFA Rules not processed for user `2df7d0fe893f4cada8ecf1e428b8814a`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:08:59.954527 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-da4eb67a-144f-45f9-aaa8-959a58371129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:09:00.013125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1420/2872] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:09:00.022818 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:09:00.025061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:09:00.083395 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad9ab0-2981-472d-8aff-ae153d3538cf admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:09:00.087489 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cd60a-6d66-46e3-9b7c-9e484a0a15f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:09:00.088149 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cd60a-6d66-46e3-9b7c-9e484a0a15f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:09:00.088437 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cd60a-6d66-46e3-9b7c-9e484a0a15f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:09:00.088712 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cd60a-6d66-46e3-9b7c-9e484a0a15f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:09:00.097774 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cd60a-6d66-46e3-9b7c-9e484a0a15f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:09:00.099186 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cd60a-6d66-46e3-9b7c-9e484a0a15f5 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:09:00.109662 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1453/2873] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:09:00.487625 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4eb67a-144f-45f9-aaa8-959a58371129 None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:09:00.489106 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-da4eb67a-144f-45f9-aaa8-959a58371129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:09:00.501576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4eb67a-144f-45f9-aaa8-959a58371129 None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:09:00.506371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:09:00.507242 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:09:00.507570 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:09:00.507847 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] PATH_INFO: `/v3/projects/03ee21bb5dad41f6b6d427201433123d` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:09:00.509125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ee21bb5dad41f6b6d427201433123d)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:09:00.511059 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:09:00.639961 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ee21bb5dad41f6b6d427201433123d', 'request_id': 'req-e12f5010-bbbd-4a83-852b-fa0070201eaa'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:00.703124 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ee21bb5dad41f6b6d427201433123d', 'request_id': 'req-e12f5010-bbbd-4a83-852b-fa0070201eaa'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:00.708872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ee21bb5dad41f6b6d427201433123d', 'request_id': 'req-e12f5010-bbbd-4a83-852b-fa0070201eaa'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:00.720402 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] The token cache is being invalidate because project 03ee21bb5dad41f6b6d427201433123d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:09:00.721033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:00.721333 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:09:00.722872 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1421/2874] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:09:00 2022] DELETE /identity/v3/projects/03ee21bb5dad41f6b6d427201433123d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:09:10.105464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:09:10.717371 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:01.576467 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cd60a-6d66-46e3-9b7c-9e484a0a15f5 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:01.585424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cd60a-6d66-46e3-9b7c-9e484a0a15f5 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:01.634127 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2fe3-33b3-46d6-93e6-ecf9ff37f291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:01.634768 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2fe3-33b3-46d6-93e6-ecf9ff37f291 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:01.635048 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2fe3-33b3-46d6-93e6-ecf9ff37f291 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:01.635320 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2fe3-33b3-46d6-93e6-ecf9ff37f291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:01.638259 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b7ff2fe3-33b3-46d6-93e6-ecf9ff37f291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:01.706672 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2fe3-33b3-46d6-93e6-ecf9ff37f291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:01.708101 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2fe3-33b3-46d6-93e6-ecf9ff37f291 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:01.719779 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1454/2875] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:02.896603 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:02.897960 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:02.911130 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f5010-bbbd-4a83-852b-fa0070201eaa admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:02.914722 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:02.918660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:02.918660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:02.918660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:02.918660 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:02.931886 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:02.933388 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:02.945068 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1422/2876] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:05.230799 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf6d9b-2f48-4e38-a725-4162a74854c4 None None] REQUEST_METHOD: `POST` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:05.231793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf6d9b-2f48-4e38-a725-4162a74854c4 None None] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:05.231793 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf6d9b-2f48-4e38-a725-4162a74854c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:05.250565 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.auth.core [None req-f8cf6d9b-2f48-4e38-a725-4162a74854c4 None None] MFA Rules not processed for user `86335999e9744e4c815a628607618b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=86792) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:10:05.251777 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f8cf6d9b-2f48-4e38-a725-4162a74854c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:05.306224 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1455/2877] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:10:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:10:12.940051 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=6 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:15.299538 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=79 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:20.614159 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:20.623085 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7077c-5e31-48ea-9d5d-db2f1a3a908b service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:20.626939 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:20.627628 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:20.627912 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:20.628229 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:20.630032 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:20.639765 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:20.641564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:20.649936 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1423/2878] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:24.890947 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf6d9b-2f48-4e38-a725-4162a74854c4 None None] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:24.892400 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-f8cf6d9b-2f48-4e38-a725-4162a74854c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:24.909316 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf6d9b-2f48-4e38-a725-4162a74854c4 None None] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:24.915545 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:24.916144 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:24.916452 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:24.916733 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:24.918550 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:24.931617 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:24.933238 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:24.944428 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1456/2879] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:30.645998 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=6 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:34.939669 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=6 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:44.142846 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:44.144409 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:44.155850 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1ad5c-c027-4134-b524-62ded3eba337 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:44.159703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:44.160380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:44.160693 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:44.160974 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:44.162849 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:44.173160 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:44.174670 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:44.183943 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1424/2880] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:45.285394 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:45.294864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61de86-a848-4f77-b6c4-6b01778f36dd service nova] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:45.298562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:45.299237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:45.299520 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:45.299795 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:45.301601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:45.316402 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:45.317943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:45.330964 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1457/2881] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:45.393782 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:45.395201 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:45.466065 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346d67d-f6b9-45a4-b0c9-8d06aa673e33 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:45.470971 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09c5cd-8d1f-4df6-b7ff-3b8760c4d697 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:45.471613 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09c5cd-8d1f-4df6-b7ff-3b8760c4d697 service placement] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:45.471894 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09c5cd-8d1f-4df6-b7ff-3b8760c4d697 service placement] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:45.472196 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09c5cd-8d1f-4df6-b7ff-3b8760c4d697 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:45.473946 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-dd09c5cd-8d1f-4df6-b7ff-3b8760c4d697 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:45.486047 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09c5cd-8d1f-4df6-b7ff-3b8760c4d697 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:45.487673 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09c5cd-8d1f-4df6-b7ff-3b8760c4d697 service placement] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:45.497863 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1425/2882] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:46.246686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:10:46.257977 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c1812a-27ae-42bc-a236-ab2834c339ed service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:10:46.264058 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:10:46.264728 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:10:46.265008 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:10:46.265282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:10:46.267059 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:10:46.279393 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:10:46.280993 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:10:46.291371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1458/2883] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:10:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:10:55.493477 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=38 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:10:56.286946 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=12 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:04.742007 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f83b6-0ba7-4585-94e5-a92c883c7c6d None None] REQUEST_METHOD: `POST` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:04.742564 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f83b6-0ba7-4585-94e5-a92c883c7c6d None None] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:04.742687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f83b6-0ba7-4585-94e5-a92c883c7c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:04.760241 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.auth.core [None req-995f83b6-0ba7-4585-94e5-a92c883c7c6d None None] MFA Rules not processed for user `86335999e9744e4c815a628607618b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=86791) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 25 06:12:04.761444 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-995f83b6-0ba7-4585-94e5-a92c883c7c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:04.768374 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1426/2884] 10.222.0.40 () {58 vars in 957 bytes} [Mon Jul 25 06:12:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 25 06:12:05.078061 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:05.088252 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b2ef5-58bf-4168-a835-edd1bee05d10 service nova] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:05.091864 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:05.092567 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:05.092854 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:05.093133 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:05.095539 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:05.104943 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:05.106371 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:05.114525 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1459/2885] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:12:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:12:05.800424 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f83b6-0ba7-4585-94e5-a92c883c7c6d None None] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:05.815269 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f83b6-0ba7-4585-94e5-a92c883c7c6d None None] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:05.821028 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:05.821696 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:05.821966 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:05.822263 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:05.824027 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:05.835447 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:05.842252 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:05.846975 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1427/2886] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:12:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:12:09.308384 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:09.323240 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e66503-a0a8-424d-96e6-dcc7855e1679 service neutron] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:09.329547 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a0f84bf3f245bdab1391a2451cb4c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:09.329696 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:09.329925 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:09.330408 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:09.332873 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:09.353453 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:09.355220 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:09.374333 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1460/2887] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:12:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:12:15.842029 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=9 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:19.363757 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=12 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:22.635143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:22.644691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae38c4-37fa-49ed-a14b-242088b26e68 service nova] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:22.648648 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:22.649329 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:22.649614 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:22.649895 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:22.651760 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:22.716576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:22.718636 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:22.728245 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1428/2888] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:12:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:12:26.640105 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:26.655027 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e7ec6-26b4-48c9-8547-7b909ae51b12 service nova] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:26.660519 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:26.661174 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:26.661455 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:26.662032 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:26.664045 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:26.677588 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:26.679623 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:26.690590 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1461/2889] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:12:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:12:30.494593 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:30.506069 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1931c6-3e8c-4f80-a540-ebf8b26b3655 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:30.511156 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:30.511806 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:30.512095 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:30.512408 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:30.514205 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:30.526320 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:30.528136 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:30.539183 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1429/2890] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:12:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:12:30.629915 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:30.632146 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:30.651237 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6462f-c2af-4229-8901-b1f62c94e3dd service placement] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:30.655899 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:30.656815 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:30.657097 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:30.657424 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] PATH_INFO: `/v3/users/3eca908cd8a1481c9e4c11036cd4df7b` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:30.658558 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eca908cd8a1481c9e4c11036cd4df7b)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:30.660589 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:30.846159 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eca908cd8a1481c9e4c11036cd4df7b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:30.913063 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eca908cd8a1481c9e4c11036cd4df7b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:30.913717 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eca908cd8a1481c9e4c11036cd4df7b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:30.921158 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eca908cd8a1481c9e4c11036cd4df7b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:30.921807 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eca908cd8a1481c9e4c11036cd4df7b'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:30.939005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1462/2891] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:12:30 2022] DELETE /identity/v3/users/3eca908cd8a1481c9e4c11036cd4df7b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:12:31.159882 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:31.170996 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713538c-4ec4-4a1d-8aad-36a148ca136a service placement] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:31.228373 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:31.229246 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:31.229530 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:31.229808 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] PATH_INFO: `/v3/projects/3520876da8974c439cef1b35397029f9` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:31.231017 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3520876da8974c439cef1b35397029f9)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:31.232982 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:31.311790 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3520876da8974c439cef1b35397029f9', 'request_id': 'req-58a83802-2141-44af-bef1-569868b4188f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.375060 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3520876da8974c439cef1b35397029f9', 'request_id': 'req-58a83802-2141-44af-bef1-569868b4188f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.381380 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3520876da8974c439cef1b35397029f9', 'request_id': 'req-58a83802-2141-44af-bef1-569868b4188f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.392433 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] The token cache is being invalidate because project 3520876da8974c439cef1b35397029f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:12:31.392719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.393343 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.394898 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1430/2892] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:12:31 2022] DELETE /identity/v3/projects/3520876da8974c439cef1b35397029f9 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:12:31.399968 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:31.401418 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:31.414343 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71aa1fc-c3da-435b-88c9-8dea89c302cf admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:31.470597 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:31.471392 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:31.471674 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:31.471945 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] PATH_INFO: `/v3/users/7493db8df3cc4195b950e6c952792cb5` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:31.473189 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7493db8df3cc4195b950e6c952792cb5)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:31.475215 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:31.651508 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7493db8df3cc4195b950e6c952792cb5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.723890 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7493db8df3cc4195b950e6c952792cb5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.724541 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7493db8df3cc4195b950e6c952792cb5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.730033 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:31.731944 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7493db8df3cc4195b950e6c952792cb5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.732605 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7493db8df3cc4195b950e6c952792cb5'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:31.750307 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1463/2893] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:12:31 2022] DELETE /identity/v3/users/7493db8df3cc4195b950e6c952792cb5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:12:32.033764 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:32.035243 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:32.053550 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a83802-2141-44af-bef1-569868b4188f admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:32.062319 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:32.109508 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:32.110413 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:32.110695 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:32.110992 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] PATH_INFO: `/v3/projects/e10757151e474c8cabd1675d8a9f9bdc` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:32.112304 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e10757151e474c8cabd1675d8a9f9bdc)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:32.114271 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:32.222310 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e10757151e474c8cabd1675d8a9f9bdc', 'request_id': 'req-d75be358-592b-49e6-bea4-178f20d57284'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:32.301197 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e10757151e474c8cabd1675d8a9f9bdc', 'request_id': 'req-d75be358-592b-49e6-bea4-178f20d57284'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:32.307627 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e10757151e474c8cabd1675d8a9f9bdc', 'request_id': 'req-d75be358-592b-49e6-bea4-178f20d57284'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:32.319092 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] The token cache is being invalidate because project e10757151e474c8cabd1675d8a9f9bdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:12:32.319384 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:32.326171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:12:32.326171 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1431/2894] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:12:32 2022] DELETE /identity/v3/projects/e10757151e474c8cabd1675d8a9f9bdc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:12:42.039165 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:42.315995 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:12:42.387432 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:12:42.393443 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:42.458505 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe4932-7211-4b2c-ad78-cfc67f00a1e8 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:12:42.521143 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:12:42.521700 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:12:42.522282 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:12:42.522562 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:12:42.524466 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:12:42.535571 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:12:42.537686 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:12:42.547540 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1464/2895] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:12:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:12:52.543244 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=58 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:06.515532 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:06.518691 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:06.573703 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75be358-592b-49e6-bea4-178f20d57284 admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:06.578072 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b1638c28af8240439d061a3ae0ddc38d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:06.578719 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] REQUEST_METHOD: `GET` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:06.579003 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:06.579280 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:06.581083 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:06.643125 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:06.644617 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:06.654544 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1432/2896] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:14:13.576713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:13.578175 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:13.592554 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b945e-bdbd-4ae0-a46c-8a141d50d276 service placement] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:13.597202 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:13.598098 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:13.598380 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:13.598652 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] PATH_INFO: `/v3/users/86335999e9744e4c815a628607618b4f` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:13.599866 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86335999e9744e4c815a628607618b4f)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:13.601871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:13.776874 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86335999e9744e4c815a628607618b4f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:13.830871 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86335999e9744e4c815a628607618b4f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:13.831484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86335999e9744e4c815a628607618b4f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:13.838484 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86335999e9744e4c815a628607618b4f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:13.839100 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86335999e9744e4c815a628607618b4f'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:13.856351 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1465/2897] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:14:13 2022] DELETE /identity/v3/users/86335999e9744e4c815a628607618b4f => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:14:14.109262 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:14.119475 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78751c3c-1c53-4900-860e-0012e5e42d92 service neutron] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:14.178260 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:14.179143 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:14.179430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:14.179710 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] PATH_INFO: `/v3/projects/d275686dbcfc44769065d5d9ba1a6f0c` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:14.180952 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d275686dbcfc44769065d5d9ba1a6f0c)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:14.182867 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:14.295215 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd275686dbcfc44769065d5d9ba1a6f0c', 'request_id': 'req-fa805037-87b2-44aa-94b6-57baa93f539c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.297684 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:14.396676 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd275686dbcfc44769065d5d9ba1a6f0c', 'request_id': 'req-fa805037-87b2-44aa-94b6-57baa93f539c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.402430 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd275686dbcfc44769065d5d9ba1a6f0c', 'request_id': 'req-fa805037-87b2-44aa-94b6-57baa93f539c'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.414296 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] The token cache is being invalidate because project d275686dbcfc44769065d5d9ba1a6f0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:14:14.414582 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.415212 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.416783 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1433/2898] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:14:14 2022] DELETE /identity/v3/projects/d275686dbcfc44769065d5d9ba1a6f0c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 25 06:14:14.420767 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:14.422170 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:14.435702 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607df15-e5ce-4ad2-91a4-7becf6df6d64 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:14.493106 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:14.493983 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:14.494294 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:14.494598 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] PATH_INFO: `/v3/users/ee69b3f14a814f0aaa4de380bf2cc819` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:14.495787 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee69b3f14a814f0aaa4de380bf2cc819)` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:14.497937 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:14.640998 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee69b3f14a814f0aaa4de380bf2cc819'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.699754 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee69b3f14a814f0aaa4de380bf2cc819'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.700948 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee69b3f14a814f0aaa4de380bf2cc819'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.708464 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee69b3f14a814f0aaa4de380bf2cc819'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.709120 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.notifications [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee69b3f14a814f0aaa4de380bf2cc819'} {{(pid=86792) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:14.717564 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:14.728928 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1466/2899] 10.222.0.40 () {58 vars in 1223 bytes} [Mon Jul 25 06:14:14 2022] DELETE /identity/v3/users/ee69b3f14a814f0aaa4de380bf2cc819 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:14:15.009177 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] Authenticating user token {{(pid=86791) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:14:15.010687 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.fernet_utils [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86791) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:14:15.024734 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa805037-87b2-44aa-94b6-57baa93f539c admin admin] Validating token access rules against request {{(pid=86791) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:14:15.082033 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2df7d0fe893f4cada8ecf1e428b8814a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6ac2b8c3194197aedc989787229928', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86791) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:14:15.082899 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:14:15.083179 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] SCRIPT_NAME: `/identity` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:14:15.083455 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] PATH_INFO: `/v3/projects/b757855ab6ea423da9361a488f6cafd6` {{(pid=86791) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:14:15.084663 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b757855ab6ea423da9361a488f6cafd6)` {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:14:15.086576 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] RBAC: Authorization granted {{(pid=86791) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:14:15.191189 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b757855ab6ea423da9361a488f6cafd6', 'request_id': 'req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:15.236811 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b757855ab6ea423da9361a488f6cafd6', 'request_id': 'req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:15.242638 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b757855ab6ea423da9361a488f6cafd6', 'request_id': 'req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f'} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:15.254549 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] The token cache is being invalidate because project b757855ab6ea423da9361a488f6cafd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86791) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 25 06:14:15.254802 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:15.255434 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG keystone.notifications [None req-4fe7ed7c-dfb8-4abe-9dac-f8809b78084f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86791) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 25 06:14:15.257073 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: [pid: 86791|app: 0|req: 1434/2900] 10.222.0.40 () {58 vars in 1232 bytes} [Mon Jul 25 06:14:15 2022] DELETE /identity/v3/projects/b757855ab6ea423da9361a488f6cafd6 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 25 06:14:24.819364 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:14:25.251439 dv-cinder-848418-2-iscsi devstack@keystone.service[86791]: DEBUG dbcounter [-] [86791] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=86791) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 25 06:17:48.330601 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] Authenticating user token {{(pid=86792) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 25 06:17:48.332010 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:17:48.468753 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60331f-651c-4a27-a1d2-f69da7612453 admin admin] Validating token access rules against request {{(pid=86792) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 25 06:17:48.527066 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad409d0d-7fbd-4cc3-a038-12ecfbdfa4d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7c4988dc5513476786f11d08059c3d13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ca1bb14e085e4c8ab908e3880a657645', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86792) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 25 06:17:48.527720 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad409d0d-7fbd-4cc3-a038-12ecfbdfa4d3 service placement] REQUEST_METHOD: `GET` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 25 06:17:48.528005 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad409d0d-7fbd-4cc3-a038-12ecfbdfa4d3 service placement] SCRIPT_NAME: `/identity` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 25 06:17:48.528312 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad409d0d-7fbd-4cc3-a038-12ecfbdfa4d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86792) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 25 06:17:48.530581 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.fernet_utils [None req-ad409d0d-7fbd-4cc3-a038-12ecfbdfa4d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86792) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 25 06:17:48.541713 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad409d0d-7fbd-4cc3-a038-12ecfbdfa4d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 25 06:17:48.543296 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad409d0d-7fbd-4cc3-a038-12ecfbdfa4d3 service placement] RBAC: Authorization granted {{(pid=86792) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 25 06:17:48.554092 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: [pid: 86792|app: 0|req: 1467/2901] 10.222.0.40 () {60 vars in 1365 bytes} [Mon Jul 25 06:17:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 25 06:17:58.549773 dv-cinder-848418-2-iscsi devstack@keystone.service[86792]: DEBUG dbcounter [-] [86792] Writing DB stats keystone:SELECT=89 {{(pid=86792) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}